Guest blog post by Bernard Marr
With an ever-growing number of businesses turning to Big Data and analytics to generate insights, there is a greater need than ever for people with the technical skills to apply analytics to real-world problems.
Computer programming is still at the core of the skillset needed to create algorithms that can crunch through whatever structured or unstructured data is thrown at them. Certain languages have proven themselves better at this task than others. Here’s a brief overview of 10 of the most popular and widely used.
Fractal landscape simulation requires a lot of computing (this one possibly produced with MATLAB)
Julia is a relative newcomer, having existed only for a few years, however it is quickly gaining popularity with data scientists praising both its flexibility and ease of use. Although designed as a “jack of all trades” language, able to cope with any sort of application, it is thought to be particularly efficient at utilizing the power of distributed systems such as Hadoop, frequently used in Big Data.
Crowd-sourced data science website Kaggle is currently running a competition which doubles as a tutorial on getting started with Julia – it will show you how to use it to create algorithms designed to detect text characters, such as roadside graffiti, in Google Street View images.
The SAS language is the programming language behind the SAS (Statistical Analysis System) analytics platform, which has been used for statistical modelling since the 1960s and is still popular today after many years of updates and refinements. Although unlike many of the other languages mentioned here it isn’t open source, so it isn’t free, there is a free University Edition designed for learners, available here.
Python is one of the most popular open source (free) languages for working with the large and complicated datasets needed for Big Data. It has become very popular in recent years because it is both flexible and relatively easy to learn. Like most popular open source software it also has a large and active community dedicated to improving the product and making it popular with new users. A free Code Academy course will take you through the basics in 13 hours.
Like Python, R is hugely popular (one poll suggested that these two open source languages were between them used in nearly 85% of all Big Data projects) and supported by a large and helpful community. Where Python excels in simplicity and ease of use, R stands out for its raw number crunching power. Its widespread adoption means you are probably executing code written in R every day, as it was used to create algorithms behind Google, Facebook, Twitter and many other services. A free, online beginners’ course in programming R can be found here.
Although SQL is not designed for the task of handling messy, unstructured datasets of the type which Big Data often involves, there is still a need for structured, quantified data analytics in many organizations. Older and less sexy than Python or R, it was still used by 30% of organizations for their data crunching, according to one poll (the same one mentioned above!) and is a useful tool for any statistician. A free course which will teach you the basics of SQL programming is available here.
Scala is based on Java and compiled code runs on the Java Virtual Machine platform, meaning it can be run on just about any platform. Just like Java it has become popular with data scientists and statisticians thanks to its powerful number-crunching abilities, and scalability (hence the name!) A free course suitable for those with some basic experience of programming another language such as Java or Python is available here.
As the name suggests MATLAB is designed for working with matrixes which makes it very good for statistical modelling and algorithm creation. It isn’t open source so doesn’t have the volume of free community-driven support but this is alleviated somewhat by its widespread use in academia meaning that many will be introduced to it at college and if not there are ample resources online. Coursera offers Vanderbilt University’s Introduction to Programming with Matlab free of charge.
HiveQL is a query-based language for coding instructions to Apache Hive, designed to work on top of Apache Hadoop or other distributed storage platforms such as Amazon’s S3 file system. It is based on SQL, one of the oldest and most widely-used data programming languages, meaning it has been well adopted since its initial development by Facebook. It has since been passed to the Apache Foundation and given open source status. An intermediate level tutorial for those already familiar with SQL is available here.
Another Hadoop-oriented, open source system, Pig Latin is the language layer of the Apache Pig platform, which is used to create Hadoop MapReduce jobs which sort and apply mathematical functions to large, distributed datasets. Like other newer languages, users can create functions in more established languages such as Python to carry out functions which are not natively supported. An online Pig tutorial can be found here.
Go has been developed by Google and released under an open source licence. Its syntax is based on C, meaning many programmers will be familiar with it, which has aided its adoption. Although not specifically designed for statistical computing, its speed and familiarity, along with the fact it can call routines written in other languages (such as Python) to handle functions it can’t cope with itself, means it is growing in popularity for data programming. An online introduction and tutorial can be found here.
- Career: Training | Books | Cheat Sheet | Apprenticeship | Certification | Salary Surveys | Jobs
- Knowledge: Research | Competitions | Webinars | Our Book | Members Only | Search DSC
- Buzz: Business News | Announcements | Events | RSS Feeds
- Misc: Top Links | Code Snippets | External Resources | Best Blogs | Subscribe | For Bloggers
- 50 Articles about Hadoop and Related Topics
- 10 Modern Statistical Concepts Discovered by Data Scientists
- Top data science keywords on DSC
- 4 easy steps to becoming a data scientist
- 13 New Trends in Big Data and Data Science
- 22 tips for better data science
- Data Science Compared to 16 Analytic Disciplines
- How to detect spurious correlations, and how to find the real ones
- 17 short tutorials all data scientists should read (and practice)
- 10 types of data scientists
- 66 job interview questions for data scientists
- High versus low-level data science