## What is the difference between Artificial Intelligence, Machine Learning, Statistics, and Data Mining

Few day ago before I saw an interesting question on stats.stackexchange.com that got my attention for a while. After spending few minutes of readings and analyzing all answers on stack I felt writing my thoughts assuming what I would have answered if I really had too.

What is the difference between Artificial Intelligence, Machine Learning, Statistics, and Data Mining ?

Would it be accurate to say that they are 4 fields attempting to solve very similar problems but with different approaches? What exactly do they have in common and where do they differ? If there is some kind of hierarchy between them, what would it be?

I assume the author of that question is trying to get a clear picture by understanding the line of separation that distinguish each field from the other. So here is my take to explain it in a more simplified way that I ever could do.

Machine learning is a science that involves development of self-learning algorithms. These algorithms are more generic in nature that it can be applied to various domain related problems.

Data mining is a practice of applying algorithms (mostly Machine learning algorithms) with the data available from domain to solve domain related problems.

Statistics is a study of how to collect, organizes, analyze, and interpret numerical information from data. Statistics can slip into two taxonomy called Descriptive statistics and Inferential statistics. Descriptive statistics involves method of organizing, summering and picturing information from data. Inferential statistics invokes method of using information from sample to draw conclusion about the population.

Machine learning uses statistics (mostly inferential statistics) to develop self learning algorithms.

Data mining uses statistics (mostly Descriptive statistics) on results obtained from algorithms, it used to solve the problem.

Data mining as a field emerged to solve problems in the miscellaneous domain (particularity in business), acquired different techniques and practices that are used in different field of studies.

In 1960 practitioners who solved problems (mostly business problems) used term Data fishing to call the work they do. In 1989 Gregory Piatetsky Shapiro used term knowledge Discovery in the Database (KDD). In 1990 a company used term Data mining with the trademark to represent their work. Today data mining and KDD are used interchangeably.

Artificial Intelligence is a science to develop a system or software to mimic human to respond and behave in a circumference. As field with extremely broad scope, AI has defined its goal into multiple chunks. Later each chuck has become a separate field of study to solve its problem.

Here is a major list of AI goal (a.k.a. AI problems)

1. Reasoning
2. Knowledge representation
3. Automated planning and scheduling
4. Machine learning
5. Natural language processing
6. Computer vision
7. Robotics
8. General intelligence, or strong AI

As mentioned in the list Machine learning is field emerged from one the AI goal to help machine or software to learn on it own to solve problems it’s can come across.

Natural language processing is another such field emerged from AI goal to help machine to communicate with real human.

Computer vision is a field emerged from AI goal to identify and distinguish objects that the machine could see.

Robotics is a field emerged from AI goal to give a physical appearance for a machine to do physical actions.

Is some kind of hierarchy between them, what would it be?

One way of representing the hierarchical relationship between these science and study can be drawn from historical facts when they have emerged.

Origin of science and study.

Statistics – 1749
Artificial Intelligence – 1940
Machine leaning – 1946
Data mining – 1980

History of statistics is believed to be started around 1749 to represent information. Practitioners use statistics to represent the economic status of states and to represent the material resource put on the military use. Later usage of statistics was leveraged to include data analysis and organization.

History of Artificial Intelligence happened to be existing has two types namely classic and modern. Classical Artificial Intelligence can be seen in ancient time stories and writings. However Modern AI emerged in 1940 when describing the idea of mimicking human like machine.

In 1946, Origin of Machine leaning emerged as branch of Artificial Intelligence with purpose to solve the goal of making machines to learning itself without programming/ hardwiring explicitly.

Would it be accurate to say that they are 4 fields attempting to solve very similar problems but with different approaches?

It would be appropriate to say they (Statistics, Artificial Intelligence and Machine Leaning) are highly inter dependent field that they can’t survive along without leading help from others. It is also good to see these 3 fields a one globe field instead of 3 diffident subjects.

As with this perception as one globe field these three fields have contributed their excellence in solving common goal. As a result the solution as such where applicable in many different domains where the core problem is same under the hood.

This is time data mining come into picture, it took the solution obtained from the globe field and applied it to different domains(business, military, medicine, space) to solve problems that has the same nature under the hood. This is also the time where data mining expanded its popularity.

I Hope my explanation has everything that need to answer the authors question and I believed it would have definitely helped anyone who is trying to understand the sweet spot of each field and how they are related. If you got anything to say or share about the article then please let me know your thoughts in the comment section.

1. Reenie Mahajan wrote
at 10:22 AM - 26th June 2014 Permalink

Great article, thank you for posting.

2. Zidanet wrote
at 8:51 PM - 26th June 2014 Permalink

3. teodorz wrote
at 3:17 PM - 26th June 2014 Permalink

Great article, thank you for posting. Do you think it’s fair to say that Data science (as in Data Scientist role) includes Statistics, ML, and Data Mining?

4. shakthydoss wrote
at 3:27 PM - 26th June 2014 Permalink

Data Scientist is just a creative designation given to a ML / DM programmer who also has knowledge about the domain (ex: medicine or space). From my point of view data science should not be considered as a separate field of study otherwise it will would be an another Data mining.

5. mattrepl wrote
at 3:18 PM - 26th June 2014 Permalink

The difference between data mining and machine learning can be fuzzy. Compare papers accepted at KDD to those at ICML. You’ll find lots of statistical inference, neural nets, and other ML techniques at both. The main difference I’ve noticed is that KDD papers tend to have more experiments with real world datasets.

6. shakthydoss wrote
at 3:28 PM - 26th June 2014 Permalink

I believe you must have definitely attended a workshop given at KDD 🙂 And yes KDD papers are good.

7. mattrepl wrote
at 3:31 PM - 26th June 2014 Permalink

Hah, that’s a good point. The papers on topics interesting to me (dynamic networks, unsupervised learning, and PGMs) tend to be similar in the ML and DM conferences, but that’s only a sliver of a large conference like KDD.

8. CageyWeasel wrote
at 3:20 PM - 26th June 2014 Permalink

To go a level higher, I’d argue these fields also overlap a lot with neuroscience, cognitive science, psychology, and economics, especially when you start talking about supervised learning algorithms that attempt to maximize a reward.

These fields each have different names for the same phenomena.

9. shakthydoss wrote
at 3:27 PM - 26th June 2014 Permalink

Totally agreed. When you go deep you get more interesting results.

10. MeowMeowFuckingMeow wrote
at 3:21 PM - 26th June 2014 Permalink

The pay

11. Zidanet wrote
at 3:21 PM - 26th June 2014 Permalink

12. shakthydoss wrote
at 8:57 PM - 26th June 2014 Permalink

Totally agreed. When you go deep you get more interesting results.

13. shakthydoss wrote
at 8:57 PM - 26th June 2014 Permalink

Data Scientist is just a creative designation given to a ML / DM programmer who also has knowledge about the domain (ex: medicine or space). From my point of view data science should not be considered as a separate field of study otherwise it will would be an another Data mining.

14. shakthydoss wrote
at 8:58 PM - 26th June 2014 Permalink

I believe you must have definitely attended a workshop given at KDD 🙂 And yes KDD papers are good.

15. mattrepl wrote
at 9:01 PM - 26th June 2014 Permalink

Hah, that’s a good point. The papers on topics interesting to me (dynamic networks, unsupervised learning, and PGMs) tend to be similar in the ML and DM conferences, but that’s only a sliver of a large conference like KDD.

16. scritic wrote
at 8:49 PM - 27th June 2014 Permalink

I’m curious why 1946 is the year for the origin of machine learning. Is there any specific development that you are referring to? Some paper, manifesto that was published?

17. shakthydoss wrote
at 9:12 PM - 27th June 2014 Permalink

Hi ,
http://www.mlplatform.nl/what-is-machine-learning/
You can also search for ENIAC and ELIZA for more reference

18. scritic wrote
at 3:19 PM - 27th June 2014 Permalink

I’m curious why 1946 is the year for the origin of machine learning. Is there any specific development that you are referring to? Some paper, manifesto that was published?

19. shakthydoss wrote
at 3:42 PM - 27th June 2014 Permalink

Hi ,
http://www.mlplatform.nl/what-is-machine-learning/
You can also search for ENIAC and ELIZA for more reference

20. Poornima wrote
at 11:54 PM - 14th September 2014 Permalink

Simple and clear. Very good article.thank you

21. jleandro wrote
at 2:38 AM - 15th September 2014 Permalink

What about Pattern Recognition? Is it simply an alias for Classification (logistic regression for categorical variables), as Murphy suggests in his book (Machine Learning a Probabilistic Perspective, 2012)?

22. Bharatranjan Kavuluri wrote
at 3:09 AM - 15th September 2014 Permalink

“It would be appropriate to say they (Statistics, Artificial Intelligence and Machine Leaning) are highly inter dependent field that they can’t survive along without leading help from others.”

I’m pretty sure that the above line is incorrect and is logically inconsistent with the remaining article.

AI can be achieved with pure logic based inference- a system can be taught to learn without the aid of statistics, if there is a human involved in the loop.

Similarly, statistics as a field can survive (as it did all those years) without AI. The advent of computers has contributed a lot of the scale of problems at which statistics can be applied and improved the accuracy, but it would be wrong to say it would not survive/lose importance without the remaining fields.

However, Machine learning – as in statistical machine learning – depends on all three of the above fields (Algorithms, Statistics and DM) for its origins and continued existence as you pointed out somewhere in the article.

I would say with my limited knowledge that the fields could be arranged in the following way, as one branch of the AI D(A?)G.

AI
———
ML
____
Pattern Recognition, Inference
—————————-
Datamining
—————————-

Algorithms, Statistics

__________

Logic,Math

This is just one path towards AI.
Without any(even one) of the bases, (in my opinion) ML won’t be able to achieve its full potential(AI and beyond). But the bases can be used for other ends too.

23. shakthydoss wrote
at 5:13 AM - 15th September 2014 Permalink

There is no 100% successful AI build using logic based inference alone.

AI in olden days and AI in modern day is totally different.

Statistics, Artificial Intelligence and Machine Leaning Highly inter dependent accepted.
Article discuss about the differences in these studies based on existence of the new study that have emerged to help the other.

Every engineer has their own though about these field, I appreciate yours as well.
Thanks

24. Bharatranjan Kavuluri wrote
at 9:09 AM - 15th September 2014 Permalink

There is no 100% successful AI (IMO), with or without statistics. Change the task for example and most of the “AI” fail. So that line of argument can not be acceptable.

Statistics based models are useful in domains where reasoning from first principles either can not be done or is too computationally intensive to be done.

I would like to correct my wrong thoughts instead of just having own thoughts 😉 that why I air/write them out.

cheers,
Bharat.

25. vineetha wrote
at 4:57 AM - 15th September 2014 Permalink

Great Article.

26. Brian Forbes wrote
at 1:36 PM - 15th September 2014 Permalink

To throw my 2 cents in here – statistics is obviously the oldest and most well understood area, and there are actual theorems with proofs. ‘Machine learning’ is a bit of a misnomer, as machines to not actually learn anything; they simply read more data and update whatever algorithm they’re running. This and artificial intelligence have (largely) simply reinvented statistics, in a suboptimal form, though many practitioners seem not to have a good grasp of statistics, so they don’t realize they are reinventing the wheel. One useful insight of the machine learning school however is nonparametric methods; whereas statistical theorems rely on making assumptions on underlying distributions, machine learning attempts to create a function which most accurately predicts the output variable, without restricting how data may be generated.

at 2:56 PM - 1st November 2014 Permalink

Great Article ! Had been struggling to understand the relation between these 4 fields.

28. shakthydoss wrote
at 3:00 PM - 1st November 2014 Permalink

I am glad. It helped you..

29. kishore wrote
at 10:00 AM - 10th November 2014 Permalink

Big Thanks .. wonderful article i have ever read . It was interesting and needed

30. shakthydoss wrote
at 2:00 PM - 10th November 2014 Permalink

Thanks kishore. I am glad you liked it.

31. liemlionel wrote
at 12:34 AM - 21st December 2014 Permalink

Machine learners are just a bunch of copycats reinventing the wheel and coming up with new names to make themselves sound cool. Logistic regression and generalized regression existed long before all that perceptron nonsense came about. Same with PCA, LDA,K-NN, trees and so many algorithms that machine learners claim as their own.

Even the more recent discoveries such as random forests and bagging are discovered by Leo Breiman, a statistician.

32. dang ky chu ky so wrote
at 10:34 AM - 9th September 2016 Permalink

Hi there, i read your blog occasionally and i own a similar one and i was just curious if you get a
lot of spam feedback? If so how do you prevent it,
any plugin or anything you can suggest? I get so much lately
it’s driving me crazy so any support is very much appreciated.

33. Kursus Pemrograman wrote
at 10:21 PM - 12th September 2016 Permalink

Just wish to say your article is as amazing.
The clearness to your post is simply cool and i could suppose
you are an expert on this subject. Well together with your
permission let me to clutch your feed to keep up to date
with drawing close post. Thanks a million and please keep up the gratifying work.

34. harga rak gondola jogja wrote
at 2:01 PM - 20th September 2016 Permalink

A theme like yours with a few simple adjustements would really make my blog stand out.

35. shakthydoss wrote
at 11:46 PM - 24th September 2016 Permalink

I customize my own template from the base.

36. Clair wrote
at 12:55 AM - 16th October 2016 Permalink

This post is really interesting. I have bookmarked it. Do you allow guest
post on your blog ? I can write high quality posts for you.
Let me know.

37. Ronald90 wrote
at 11:16 PM - 20th October 2016 Permalink