Exploit the power of deep learning with IBM Watson Machine Learning

Artificial intelligence is older than the Internet, while the term “artificial intelligence” was coined for the 1956 Dartmouth College Summer Research Workshop. Since then, artificial intelligence has occasionally, but always strikingly, come to the center of attention. Nowadays, we use artificial intelligence on a daily basis, and there is almost no industry in which artificial intelligence is not used. It is mainly about business transformation, improving the user experience and creating innovations. Artificial intelligence is, to put it simply, all those systems that, as it is commonly said, behave in some way similar to humans. However, this does not mean that these systems are self-aware or similar to human intelligence; it just means that the system is capable of solving a particular problem.

Artificial intelligence is not there to replace humans, but to help them make more informed decisions and make it easier for them to perform certain tasks. The concepts of machine and deep learning are associated with artificial intelligence. Part of the algorithms within artificial intelligence belongs to machine learning, while deep learning deals with problems within machine learning whose dimensionality is much larger. Machine learning deals with the design of algorithms that improve their efficiency based on empirical data and are, as time passes, able to improve their work.

The rapid growth of deep learning

Deep learning is the fastest growing subcategory of machine learning that uses neural networks to try to develop a system for recognizing certain patterns. Today, various companies use deep learning to develop powerful algorithms that cover the areas of computer vision, natural language processing, speech and sound recognition, bioinformatics and many other current areas. One of the most well-known applications of deep learning today is the recognition of objects in images. For example, one of the use cases is the automatic control of wearing protective equipment in workplaces that require it, and in Megatrend poslovna rješenja we have developed such a solution for automatic detection of protective helmets and vests.

Data science deals with all of the above. It is important to emphasize that data science is a team sport. So, in order to make a solution that will start from the development and end in the phase of production and constant improvement of the product, you need a lot of different types of knowledge and skills. All this knowledge is difficult to find in one person, so it is important to put together a team of different profiles of people with appropriate skills. Starting with a data engineer, who knows the data sources and who will collect data, process and prepare them for a data scientist who will continue working with this data to create and train models, to a business analyst who will know how to further interpret model results and finally DevOps engineer and development engineer who will allow that model to be put into production or included within the application.

The challenges of deep learning projects

The development of such solutions brings certain challenges, such as the installation and start-up of the system, which can take weeks or months, which, although being one-time, is a great waste of time. In deep learning, the emphasis is on a large amount of quality data, and the processing and collection of it takes a lot of time. Then, the training and optimization of machine and deep learning models also represent a large computational load and cost of time resources. If we find a solution to save time, we are able to reallocate it to some other useful tasks. IBM’s Watson Machine Learning tool helps with just that. It is a platform that creates an environment with all the necessary functionalities for the development of deep learning models, with which we are able to develop fast and accurate models in a relatively short time.

IBM Watson Machine Learning (WML) is enterprise-ready which means it is designed according to the requirements and needs of enterprise companies. In other words, all the necessary security, authentication, authorization and everything else is included in order to spend as little time as possible on installation and setup, and you get an environment ready to work according to all standards. Also, in this way, the enterprise-ready AI environment is available to everyone for the first time.

It is important to emphasize that WML is built from open source tools. The vast majority of tools used by data scientists in creating machine and deep learning models come in open source variants, and one of the advantages of this is that it is possible to start working without the initial cost of licenses. Nevertheless, merging and configuring multiple open source tools is a demanding job where errors are possible and the time spent can be significant. However, WML comes with all the necessary tools and the latest ready-to-run versions installed, and focuses on simplifying access to versions of the various environments used in machine and deep learning.

IBM Watson Machine Learning performances

WML performance takes full advantage of IBM Power architecture and specially designed elements specific to AI processes. It is a computer that was initially designed to reach its maximum by working with machine and deep learning models, making them and training them and all the other processes needed to make the whole process work. It is an architecture that enables the linear growth of machine learning systems without performance difficulties and without the need to redesign the system. Such performance is enabled by graphics cards. It is important to emphasize that NVLink is used between the GPU and the CPU, which speeds up communication between the graphics card and the processor, and such a complex system allows not only speed but also the use of large and complex models. This allows you to cross the memory limit of graphics cards, which in turn means the use of much larger models and, when it comes to image recognition, the use of high-resolution images. Most models used for image recognition work on a set of data (images) with reduced resolution, which is sometimes enough, but there are also cases when the details decide on the accuracy of the model and then the images of higher resolutions must be used.

WML comes in two variants, as the Community Edition, which is based entirely on open source technology, and the IBM Watson Machine Learning Accelerator, which includes some additional functionalities. WML is also available as part of the IBM Cloud Pak for Data platform. The IBM Watson Machine Learning Accelerator includes a variety of IBM system scaling and management tools. Included are IBM Spectrum LSF (Load Sharing Facility), which consists of one of the fastest and most reliable HPCs (High Performance Computing) on ​​the market, and IBM Spectrum Conductor for automatic process management and performance and resource optimization. It also includes the following technical components: multitenant architecture, Elastic Distributed Training, which allows for dynamic distribution of tasks by resources, parallel optimization of hyperparameters, visualization during the model training process and scaling from one to multiple servers. The IBM Watson Machine Learning Accelerator also comes with the SnapML package, which allows for very fast training of machine learning models on GPU/CPU systems.

Conclusion

With this platform, IBM has created a solution that enables and facilitates the collaboration of experts in various fields of artificial intelligence, all with an emphasis on speeding up the process and saving time. We can say that IBM Watson Machine Learning is a complete environment that allows for easy management of AI projects and processes from start to finish, i.e. production.