Learnit Training
link]

Web scraping with Python: data at your fingertips

Everyone uses them: websites where you can compare different products or services. Want to book a hotel? Click on the arrival date and fill in the destination on a site like expedia.co.uk which compares different hotels and prices of suppliers. Looking for new trainers? Type in trainers at beslist.nl and enter, for example, what size you are and what colour trainers you are looking for. This way you'll be able to find your perfect trainers. Handy anyway! The technique that makes it possible to search in this fast way, is also called webscraping.

What is web scraping?

Webscraping enables people to retrieve important data from a site or to search for certain characteristics on a website. Data can be retrieved or stored automatically and periodically. Website managers can use various programming languages to enable web scraping, including Python. The 'Beautiful Soup' module is very useful for web scraping, and 'numphy' and 'pandas' make analysis much easier. Webscraping ensures that visitors to a comparison website, for example, can easily and quickly find the information they require. Useful for websites like expedia.comExpedia, which actually knows how to make smart use of the data and information of other websites. Expedia attracts customers, as it were, because the company is able to bundle data from other websites in a smart way and display the page in such a way that it is attractive to consumers looking for a hotel.

Benefits of web scraping

The most important advantages of web scraping are listed below:

  • Automatically retrieve important data from a website
  • Easy way to get more insights through data analysis
  • Store usable data easily and quickly

In short: webscraping is useful for anyone who wants to learn how to get the most out of processing data from their own website!

Webinar and Training Python

Are you interested in webscraping and do you want to follow a short introduction webinar about it? Then click here to watch the past webinar of expert trainer André via our YouTube channel. Did you know that Learnit, next to webinars, also offers various trainings in Python on different levels? When you have no or little experience with Python, our training course Python programming basics is really something for you. In this three-day training, you are introduced to the basic concepts and functions in Python. Do you already have experience with programming in Python? Then the training Python programming continuation Then the training is something for you! To view our complete overview of training courses related to Python, please click here.

link]

Top 10 most popular programming languages

At the end of last year, a list of the most popular programming languages of 2020 was announced. Not entirely surprisingly, as ultimate favourite, C was still on top of TIOBE's index. This year, Python did manage to surprise us: for the first time in history, Python overtook Java and we find Python on place 2 and Java on place 3 in the top 10. Besides Python, also R, Perl and Go are gaining popularity. R even ended surprisingly on place 9 in the top 10 list; TIOBE has therefore labelled R as 'programming language of the year 2020'.

Why is Python gaining popularity?

The CEO of TIOBE's INDEX Paul Jansen said about Python catching up with Java: "Some time ago I had a flat tire and called the road patrol to help me out. The mechanic asked about my living and when I used the word 'software' in my answer, he smiled and started talking very enthusiastically about his own passion: programming in Python. From that moment on, I knew Python would become ubiquitous." Whereas ten years ago it was mainly software engineers who were concerned with the various programming languages, nowadays you see that more and more people on the shop floor have programming skills. This has everything to do with the fact that there is a shortage of classical programmers. Moreover, a programming language such as Python is relatively easy to use when you want to work with large quantities of data. Would you like to work with data yourself, but do you have little or no knowledge of a programming language? Then take a look at our courses Python programming basics or Python for data analysis. Do you already have a fair knowledge of Python? Then the course Python programming continuation course is really something for you!

Why is 'R' gaining popularity?

Big data is a term that most people undoubtedly hear more than once. Programmes such as R are perfect for converting these so-called big data into useful information for organisations, and R is also a lot easier to use than a language such as 'C'. For example, did you know that R is one of the programming languages that was used to find a COVID-19 vaccine? In addition, you can see that universities and colleges are increasingly exchanging static programmes such as Stata, SAS and SPSS for static programming languages such as R. Professional groups in which many static analyses occur are also increasingly using R. Think for example of data analysts, mathematicians, econometricians but also marketers. The advantage of a programme like R is that it costs nothing: it is an 'open source' project, which anyone can use and improve. In addition, R is known for the fact that you can easily visualise data with packages such as 'ggplot2', with which you can make a graph, for example.

link]

Practical examples of the Python programming language

In his previous blog, trainer André showed that the programming language Python is used all around us. From the pictures that telescopes take to Netflix and Dropbox, Python often plays an important role. In this blog, André will show by means of examples how you can use Python in practice.

Excel

Adjusting data in an Excel file, adding new columns, discovering a trend in a series of data, removing errors from data and filling in missing data. You can do all this very easily and quickly with a Python programme. In addition, you can save the settings so that you can easily reuse them in the same way.

Python packages

Millions of people around the world use Python, a free language with many packages. The packages numpy, pandas and pyplot are important for fast and efficient data editing. With numpy, data can be edited quickly, pandas is very suitable for editing tables and with pyplot, you can make graphs of the data.

Imagine that a long list of numbers needs to be multiplied by a number. You do that in numpy as shown below.

You do not have to multiply each number by 20. That happens in one go.

As you can see below, you can also easily add up lists of numbers.

Pandas allows you to extract, edit and write back data from an excel file. Let's take a small excel file with winning and losing points in a competition as an example.

We read these data with pandas.

Then we add up the points with pandas.

After this we can write data back to the excel file.

The file now looks like this.

In the last example you can see that it is also easy to make a graph of the table.

The conclusion we can draw from these examples is that you can use Python in many ways to make your work a lot easier. The examples are of course only a small part of the various possibilities that Python offers.

Did this blog pique your interest? Then make sure you watch the free webinar of 16 January. André discussed in half an hour how easy it is to use Python in practice. Watch it here!

link]

Python is everywhere

In his previous blog, trainer André showed that you can quickly create a program in Python that does useful things: read and analyse a file, edit data and then make a graph of it. In this blog, the trainer shows where Python is used.

Black hole

The image above shows a black hole. The image was taken by eight telescopes - spread around the world during seven days in 2017. The telescopes received data from a black hole 55 million light years away.

Python came into play when processing that data. Freely accessible and available modules of the programming language, such as Numpy and Pandas, helped to represent the black hole. Click here for more information.

Netflix

Streaming service Netflix also uses Python. The company uses the programming language to improve video streaming. Python records which network devices are active. In this way, the language registers changes in the network, in order to analyse whether a device is still functioning properly.

The streaming service also uses Python to ensure that the geographic storage of content is organised in such a way that a video is available to a user as quickly as possible. In this tech blog about Netflix, you will find even more applications for which Python is used.

Dropbox

The popular tool Dropbox uses Python for almost all of its functions: storage, sharing, synchronisation, permission management and more. The company bought Guido van Rossum, the creator of Python, from Google in 2012. As a result, Dropbox has had the best possible support for Python ever since.

Frameworks

Several web frameworks have been written for Python. One of the most popular is Django. This framework, originally written for a well-known American newspaper, has developed into a universally applicable framework.

Django makes it easy to create a website. Python makes it easy to write lines of code and Django gives you structure without much effort. This allows you to maintain your website and expand it, for example.

Screenshot from www.nationalgeographic.com, a website built with Django.

One very notable application of Python is the programming language's contribution to Artificial Intelligence (AI). Google is very active in the field of AI and uses Python to control its AI network.

The Python package that the company has developed for this is called Pytorch. Setting up an AI network is not easy. Many other ways of setting up such a network require a large amount of data to be recorded beforehand. Pytorch makes it possible to set up the network while creating the AI.

In short: more and more companies and organisations use Python, because the language is easy to learn and because there are many modules available for Python to do what they find important. So if you are interested in your own future, then immerse yourself in Python.

Free webinar

André gave a webinar on Python on 16 July 2019. ''I'm mainly going to show there how easy Python is to use,'' the trainer said. Watch it back here.

link]

Python: The language for everyone

Python is a programming language in which everyone can quickly learn to program. With this language, you can relatively easily make a small programme, develop a website, or make an application. Python is also often used for artificial intelligence, or for analysing data. In short, Python is everywhere.

It is useful to know that Python was invented in 1989 by a Dutchman, Guido van Rossum. Van Rossum was working at the Centre for Mathematics and Computer Science in Amsterdam and wanted to write programs in a simple way for a project that was being carried out at the institute.

Growing interest

Although the language was initially only used by a handful of people, interest quickly grew. Due to the expert guidance of Van Rossum, the use only increased. The Dutch polder mentality has ensured that the user group of Python is friendly and open, for everyone who wants to work with Python.

Today, there are thousands of Python files, called modules, that allow the user to write the software he needs. In Python, for example, it is easy to open a text file and count which words appear in it and how often. Then, in a few Python sentences, the user can extract the most common words and make a graph of them.

Analysing with Python

Even downloading data from a website can be done in a few Python lines. Python is widely used for data analysis. For example, a company wants to make an analysis of the usefulness of the advertisements it places on various local media.

For this, a link must be made between the area where the media appear and the sales in the shops there. Here too, Python can help. In a few lines there is an analysis, and if you want also graphs. If it is useful to put the data into an Excel spreadsheet, that only takes one line of Python.

Python is also widely used for scientific computing and artificial intelligence. All artificial intelligence platforms have been made suitable for use with Python and because the language is so easy to learn, many scientists and engineers use it.

The company Tiobe keeps track of which language is widely used. Python is in third place in this ranking. It is very likely that Python will rise even further.

The conclusion is that if you need programming in your job, Python is the language to learn. In just a few days, you can write small but useful programs to analyse website data, for example, or to monitor the use of network equipment. link]

You can learn to program!

Visit any job site and you will see it immediately: there is an unprecedented demand for programmers. If you are looking for a well-paid job with excellent future prospects, then learning to program is a must. Even for jobs where you do not program yourself, a programming language is an excellent addition to your CV. However, there are countless programming languages, so where is the best place to start? And which languages are interesting for the future? We made an overview of the 10 most important programming languages. Languages in which you can also receive training immediately: let those jobs come!

Java

Java was the most popular programming language for employers for many years. Although it has recently been surpassed by SQL, it is still very popular. Java is the standard for many enterprise software, is used by more than 9 million developers and is used to control more than 7 billion (!) devices worldwide. The programming language owes its popularity mainly to the simplicity and readability of the code, which makes it a good starting point for beginning programmers. Also, one of Java's basic principles is compatibility, which ensures that older applications continue to work.

SQL

You will find SQL everywhere. Database technologies such as MySQL, PostgreSQL and MicrosoftSQL keep banks, universities, hospitals, small and large companies up and running. It is no exaggeration to say that every computer and person with access to technology has come into contact with SQL in one way or another. For example, iOS and Android devices have access to SQLite, an SQL database, and there are countless mobile applications that use SQL directly.

C

C dates back to the early 1970s, when it was developed as a generic programming language. As one of the oldest programming languages in existence, C provides the building blocks for other popular languages such as C++, Java, Javascript and Python. C itself is mainly used for operating systems and industrial applications.

C++

C is an object-oriented programming language based on C. Since the language was developed in the 1980s, it has been used to develop Adobe and Microsoft applications, MongoDB databases and large parts of Mac OS/X operating system. It is considered by many to be the best programming language to learn for applications where performance is of great importance, such as games or audio/video editing.

iOS / Objective C / Swift

Until 2014, Objective C was the programming language used by the Apple operating system. It plays a role in Apple's OS X, iOS and its APIs. For a long time it was the language in which you as a programmer had to write iPhone applications, until Apple came up with its own programming language a few years ago: Swift. Swift is very similar in many ways to C++ and its predecessor Objective C, but is a little easier to learn for most programmers.

Javascript

Javascript - not to be confused with Java - is widely used to add interactivity to websites and is supported by all major internet browsers. Not only can it be used for simple functionality such as pop-ups or simple games, it also forms the basis for Node.js - a server technology widely used in real-time communication.

PHP

PHP was actually never intended to be a programming language. Danish-Canadian programmer Rasmus Lerdorf developed PHP in 1994 as a set of tools that enabled him to manage his personal website (Personal HomePage). Today, the meaning of the abbreviation has changed to Hypertest Pre-Processor and the language is used on servers to generate HTML websites dynamically. PHP is easy to learn, but has many advanced features that are of great value to experienced programmers.

HTML / CSS

HTML forms the basis for web pages. The programming language does this by basically laying down a structure that can be extended or adapted by other languages. For example, Javascript can be used to add advanced interactivity and CSS influences the presentation, formatting and layout of web pages. Although PHP can generate HTML from a server, knowledge of HTML and CSS is an absolute must for web developers.

Python

Python is a server-side programming language for websites and mobile applications. It is a relatively easy language to learn, thanks to the readability and conciseness of the code. In Python, it is often possible to express a concept in fewer lines than in many other languages. Python is used in applications such as Instagram and Pinterest in the form of the Django web framework. Google, Yahoo! and NASA all use Python.

Ruby

Ruby is - like Python - a programming language that is very user-friendly for beginning programmers. At the same time, the object-oriented language is very dynamic and therefore popular for developing websites and mobile apps. It is the basis of the Ruby on Rails framework used by Groupon, Shopify, Scribd and GitHub.

Complete the list!

Of course, there are many more programming languages. We have listed just a few of the most popular ones for you. Do you have any additions? Are you missing a language that you use really often? Share your knowledge and experience with us!

Want to learn a programming language?

Do you want to learn a (different) programming language? Then quickly discover our workshops and courses. Under the guidance of an experienced trainer, you will learn the basics of programming in an efficient manner. Are you a bit further on? Then it is possible to register for an advanced training. You will then go deeper into the subject matter. Such a training course is not only fun, but it is also closely linked to practice. That way, you know for sure that you can put your new knowledge and skills into practice immediately!