Written by

Bernard Marr

Bernard Marr is a world-renowned futurist, influencer and thought leader in the fields of business and technology, with a passion for using technology for the good of humanity. He is a best-selling author of 20 books, writes a regular column for Forbes and advises and coaches many of the world’s best-known organisations. He has over 2 million social media followers, 1 million newsletter subscribers and was ranked by LinkedIn as one of the top 5 business influencers in the world and the No 1 influencer in the UK.

Bernard’s latest book is ‘Business Trends in Practice: The 25+ Trends That Are Redefining Organisations’

View Latest Book

Big Data: What is Python – An Easy Explanation For Absolutely Anyone

2 July 2021

Here is another post in which I try to disentangle some of the concepts that underpin today’s big data world. In this post I look at Python, which is an open source programming language commonly used for data manipulation in commercial Big Data operations.

Python is a programming language frequently used to create algorithms for sorting through and analysing the huge amounts of data collected by businesses and organisations around the world today.

In a nutshell I would say that there are three core strengths of Python which have contributed to its enthusiastic adoption by programmers working with Big Data, and they are:

Powerful libraries which mean it can easily be used to process very large, growing sets of data
Simple syntax and command set, meaning it is relatively easy to write code, and for that code to be understood by others
Strong support from users and the Open Source community, meaning it integrates very well with other open source platforms commonly used in Big Data ( Spark, Hadoop etc).

The software which allows us to create programs in Python is open source – meaning it is in the public domain and can be freely used by anyone. A big advantage of open source software is that anyone can modify it and create their own versions to do specific tasks – this is one of the main reasons that the concept of open source has been enthusiastically embraced by Big Data fans. It allows a great deal of flexibility. (See also Hadoop).

Python is a high level language – meaning that the code which the programmer types into to create the program is more like natural human language than code written to control machines. This not only makes things simpler for the programmer, it means others are more likely to understand the code if they want to use it themselves. The high-level, human-like code is converted into machine code which is understood by machines, through a piece of software known as an interpreter.

This means that programs written in Python can be run on any computer operating system which has an interpreter for it – which is pretty much all of the operating systems you are ever likely to come across! This means that code can be ported between projects and organisations even if the people running it are using completely different hardware (as is often the case in projects using open source technologies). Because of the huge amount of support it has from the open source community, it also has very good support for a large number of file and database formats, which it can directly read from and write to.

Aside from its ease of use and portability, one of the features which has made it particularly popular with developers working on Big Data projects is the powerful libraries available for it. These are mostly extensions to the functionality that can be created in programs written in the language, and many programmers have created powerful and versatile tools and algorithms specifically designed at manipulating the large amounts of data that come with Big Data initiatives.

Another feature is that it is great for creating scalable systems – in fact it is used for creating much of the back end, data-processing functions of Google, Youtube and Facebook. As well as constantly increasing in size, these services need to be constantly updating and adding to their functionality. With giant operations such as these, programmers need an environment where new code (features) can be integrated on-the-fly without disruption of the service to users. Python is ideal for this as it is designed for use in “agile” environments where new features need to be added on-the-fly, first in a limited way for testing, and then rolled out across the entire system.

So, that’s just a quick and basic overview of what Python is, and why it’s so popular with programmers working on data projects. If you want to learn more, there are a lot of resources online, and a good place to start is Python.org (mainly written for programmers or people with some knowledge of programming conventions). If you want to learn how to use Python, there are plenty of great, free resources too, such as Code Academy and Coursera.

Data Strategy Book | Bernard Marr

Related Articles

Should I Choose Machine Learning or Big Data | Bernard Marr

Should I Choose Machine Learning or Big Data?

Big Data and Machine Learning are two exciting applications of technology that are often mentioned together in the space of the same breath [...]

What Are The Latest Trends in Data Science | Bernard Marr

What Are The Latest Trends in Data Science?

Here’s an overview of how this usage is evolving – signposts that point the direction of travel between where we are today and where data science will take us tomorrow [...]

3 Key Ways to Monetize Your Data | Bernard Marr

3 Key Ways to Monetize Your Data

I’ve written a book on data strategy, and one of my primary jobs is guiding businesses through the process of using their data effectively. [...]

The Future of Quantum Computing | Bernard Marr

The Future of Quantum Computing

A Chinese team of researchers has recently unveiled the world’s most powerful quantum computer [...]

How Facebook Is Using Artificial Intelligence

Every day, nearly 2.5 billion people log in to one of the [...]

Amazon: Using Big Data to understand customers

Amazon has thrived by adopting an “everything under one [...]

Stay up-to-date

  • Get updates straight to your inbox
  • Join my 1 million newsletter subscribers
  • Never miss any new content

Social Media



View Podcasts