Cloud-Based Machine Learning

Cloud-Based Machine Learning

Cloud-Based Machine Learning

Tanushri Kaniyar

Cloud-Based Machine Learning

Tanushri Kaniyar

ISBN - 9789361522253

COPYRIGHT © 2025 by Educohack Press. All rights reserved.

This work is protected by copyright, and all rights are reserved by the Publisher. This includes, but is not limited to, the rights to translate, reprint, reproduce, broadcast, electronically store or retrieve, and adapt the work using any methodology, whether currently known or developed in the future.

The use of general descriptive names, registered names, trademarks, service marks, or similar designations in this publication does not imply that such terms are exempt from applicable protective laws and regulations or that they are available for unrestricted use.

The Publisher, authors, and editors have taken great care to ensure the accuracy and reliability of the information presented in this publication at the time of its release. However, no explicit or implied guarantees are provided regarding the accuracy, completeness, or suitability of the content for any particular purpose.

If you identify any errors or omissions, please notify us promptly at "educohackpress@gmail.com & sales@educohackpress.com" We deeply value your feedback and will take appropriate corrective actions.

The Publisher remains neutral concerning jurisdictional claims in published maps and institutional affiliations.

Published by Educohack Press, House No. 537, Delhi- 110042, INDIA

Email: educohackpress@gmail.com & sales@educohackpress.com

Cover design by Team EDUCOHACK

Preface

Machine Learning in the Cloud provides the interested with a full review of machine learning and its implementation based on the cloud. The book has been written in an easy to understand and manner to ensure readability for anyone interested in the subject. The intention behind Machine Learning in the Cloud is to help anyone who is looking to find solutions to various problems through implementing machine learning using cloud-based services.

Technology has been advancing faster than we can keep up, and everyone must understand how advantageous it can be for them. Cloud services have made the machine learning process ever more convenient, profitable, and efficient. The book aims to help save time for the interested by offering all they need to know about ML and cloud services in a single place. Two things have been mostly targeted with the book – one is low readability level while the other one is to cover as much as information possible to ensure that the book is a great source. Anyone who can read will be able to read and understand the topics of the book and implement them in real life.

There are various reasons for people to be interested in this book. Firstly, a significant amount for information has been covered by the book, including practical examples, so that understandability is higher, and no information gets left out. Anyone with the Machine Learning in the Cloud can rest assured that they can get help from it when they come across any issue while working on any cloud-based machine learning work. This is the book to go to in case you are struggling with your cloud ML needs.

You won’t have to seek help from experts to get solutions for ML issues, as this book has been designed in a way that you can fix your problems all by yourself. Self-help is perhaps the most effective way of help. We hope this book will fix your ML issues and find better solutions.

Table of Contents

1 The Fourth Industrial Revolution 1

1.1 Introduction 1

1.2 The First Three Industrial Revolution 3

1.3 What is the Fourth Industrial Revolution? 6

1.4 Impacts of 4th IR on Business, People & Government 8

1.5 Challenges & Opportunities of the 4th IR 11

1.6 Technologies Assisting the 4th IR 13

1.7 AI Influence on the 4th IR 18

1.8 How to Prepare for the 4th IR? 21

1.9 Summary 23

1.10 References 25

2 The Basics of Practical AI 28

2.1 Introduction 28

2.1.1 Differences between Machine Learning

and Deep Learning 30

2.2 Practical Artificial Intelligence 31

2.2.1 Importance of Practical AI 32

2.2.2 Examples of Practical AI 33

2.2.3 Getting Started on AI 35

2.3 Development Timeline of Python 35

2.4 Practical Overview of Python 38

2.4.1 Characteristics of Python 39

2.5 Procedural Statements, Compound Statements

& Printing 41

2.6 Variable: Creation & Utilization 47

2.6.1 Creating Variables 47

2.6.2 Variable Names 47

2.6.3 Assign Value to Multiple Variables 48

2.7 Adding Numbers 49

2.7.1 Mathematical Constants 50

2.8 Number Addition & Subtraction 50

2.9 Decimal Multiplication 51

2.10 What are Strings? 51

2.10.1 How to Format Them? 52

2.11 Exponents Use & Rounding Numbers 53

2.11.1 Exponents Use 53

2.11.2 Rounding Numbers 54

2.12 Changing Numerical Types 56

2.13 Dictionaries 57

2.14 Data Structure 58

2.15 Functions & Lists 63

2.16 Python: Control Structures 64

2.17 Summary 68

2.18 References 68

3 Cloud AI Development: Google Cloud Platform & AWS 70

3.1 Introduction 70

3.2 What is Google Cloud Platform? 71

3.2.1 Different Elements of GCP 72

3.2.2 History of GCP 73

3.2.3 GCP Services 74

3.3 Colaboratory 84

3.4 What is Data Lab? 84

3.4.1 Data Lab Extension using Google & Docker Container Registry 86

3.4.2 Data Lab: Starting with Powerful Machines 86

3.4.3 Launching Powerful Machines with Data Lab 88

3.5 What is BigQuery? 89

3.5.1 Data Movement from Command Line

to BigQuery 89

3.6 AI Services Based on Google Cloud 91

3.7 Tensor 92

3.7.1 Tensorflow and Cloud TPU 92

3.8 Cloud TPUS: Running MNIST 92

3.9 AWS for Virtual Reality and Augmented Reality Solutions 93

3.10 EFS & Flask for AR/VR Pipeline 93

3.10.1 Flask, Pandas and EFS for Data Engineering Pipeline 93

3.11 Summary 94

3.12 References 94

4 AI Toolchain, ML Toolchain & Lifecycle of Spartan AI 96

4.1 Introduction 96

4.2 Data Science System of Python 98

4.3 R, Shiny, GGPlot & Rstudio 103

4.4 Google and Excel Sheets 105

4.5 AWS for Cloud AI Development 107

4.5.1 AWS: DevOps 109

4.6 Data Science: Setup of Basic Docker 111

4.6.1 Why Choose Docker? 112

4.7 Build Servers: CircleCI, Travis, and Jenkins 113

4.8 Practical Production Feedback Loop 115

4.9 AWS Batch, Glue Feedback Loop & Sagemaker 117

4.10 Feedback Loops Based on Docker 120

4.11 Summary 123

4.12 References 124

5 Intelligent Slackbot Creation on AWS 125

5.1 Introduction 125

5.2 What is an Intelligent Slackbot? 126

5.3 Creation of a Bot 129

5.4 Library to Command Line Tool Conversion 130

5.5 Bot Development using AWS Step Functions 133

5.6 Setting Up IAM Credentials & Working with Chalice 135

5.6.1 Working with Chalice 137

5.7 Step Function Building 137

5.8 Summary 139

5.9 References 141

6 NBA: Social Media Influence Prediction 142

6.1 Introduction 142

6.2 Problem Phrasing and Data Gathering 144

6.3 Data Sources Collecting Challenges 146

6.4 Athletes: Wikipedia Pageview Collection 148

6.5 Athletes: Twitter Engagement Collection 149

6.6 Data Analysis of NBA Athletes 150

6.7 NBA Players & Unsupervised Machine Learning 151

6.8 R: Faceting Cluster Plotting for NBA Players 154

6.9 Combining Data of Teams, Power, Endorsements & Players 157

6.10 Further Learnings & Practical Steps 159

6.11 Summary 161

6.12 References 163

7 Optimizing EC2 Cases on AWS 164

7.1 Introduction 164

7.2 AWS: Running Jobs and Spot Instances 165

7.2.1 Concepts 165

7.3 Real Estate Value Exploring in the US 166

7.4 Python: Interactive Data Visualization 167

7.4.1 Matplotlib 168

7.4.2 All about Bokeh 170

7.4.3 Benefits of Bokeh 172

7.5 Clustering on Price & Size Rank 175

7.5.1 The Idea of Clustering Analysis 178

7.6 Summary 183

7.7 References 185

7.8 Websites 185

8 GitHub Organization: Project Management

Insights Finding 186

8.1 Introduction 186

8.2 Software Projrct Management Issues 188

8.3 SPM Exploratory Questions 190

8.4 Project Skeleton Creation for Primary Data Science 192

8.5 Collection and Transformation of Data 195

8.6 Talking to a Whole GitHub Organization 196

8.7 Domain Specific Stats Creation 197

8.8 CLI: Wiring Data Science Projects 198

8.9 Github Organization Exploring with Jupyter Notebook 199

8.10 Pallets GitHub Project 199

8.11 The CPython Project: File Metadata Consideration 201

8.11.1 The CPython Project: Deleted Files Considerations 202

8.12 Python Package Index: Deploying a Project 203

8.13 Summary 204

8.14 Websites 205

9 Production AI: Content Generated by Users 207

9.1 Introduction 207

9.2 Production, AI Implementation & the Netflix Prize 208

9.3 Recommendation Systems: Key Concepts 209

9.3.1 Collaborative Recommendation System 209

9.3.2 Demographic Recommendation System 209

9.3.3 Utility-Based Recommendation System 211

9.3.4 Knowlegde-Based Recommendation System 211

9.3.5 Hybrid Recommendation System 211

9.4 Surprise Framework Utilization in Python 213

9.4.1 Software Framework 213

9.4.2 Python Surprise 213

9.5 Recommendation Systems Using Cloud Solutions 214

9.6 Recommendation for Real World Production 216

9.7 Integrating Using Production API 219

9.7.1 API 219

9.7.2 Alchemy API 221

9.7.3 Aylien 221

9.7.4 Lexalytics/Semantria 221

9.8 Cloud Sentiment & NLP Analysis 222

9.9 NLP on Google Cloud Platform & Azure 227

9.10 Entity API Exploration 229

9.11 AWS: Production Serverless Artificial Intelligence

for NLP 229

9.12 Summary 230

9.13 References 231

Index 232

Chapter

1 The Fourth Industrial Revolution

1.1 Introduction

Technology today has made the lives of people, incredibly easier than it was just a few decades ago. The boons of modern technology like robots, advanced computer systems, automation, etc. have revolutionized every industry. It has reached the remotest parts of the world and impacted innumerable lives. The early and later 1990s is marked as the time when the technological boom started to touch the lives of people. It gained traction within a short span of time and gave rise to biotechnology, personal computing, advanced telecommunications and a plethora of other technologies that were previously inconceivable. The pieces of the puzzle had finally fit together, and a different kind of innovation journey had been set in motion. When the first silicon microchips were introduced, computing power grew on an unprecedented level. By the time it was 2002, the first major moves to inaugurate the internet had already been made by Teledesic. And by 2005, a chunk of the world was capable of communicating over the internet. The big names in technology that are known today, like Google, Yahoo, Amazon, etc., all made their baby steps during the late 1990s or at the dawn of the 2000s. As of today, the companies have extended their services and reach far beyond imagination. Giant tech companies are at the center of the most basic everyday things. Silicon Valley has changed the landscape of the world and leveled the playing field. Automation, smart robots, driverless cars, artificial intelligence, cloud-based machine learning, etc. are just some of the many wonders that technology has brought to the world.

The term machine learning, although not new, is mostly unfamiliar to people who do not work in technical fields. It was first coined in 1959 as enabling computers to learn new things without having to reprogram them. Essentially, machine learning is a process of being able to train a computer without putting in further instructions. It is an organized set of instructions or pathways that tell a computer how to self-learn through recognizing repeating patterns and other methods. It is a means to make a computer largely self-sufficient through training so that there is minimal human intervention. The computers also learn how to make highly accurate, calculated predictions based upon past input of data. These projections from highly sophisticated systems help businesses and individuals make smarter, timelier decisions. It is capable of having a huge impact on how a company handles its business processes and, in the long run, may lead to a more profitable and sustainable business.

Cloud-based machine learning has the potential to take the businesses of today to new levels. Essentially, it is machine learning, only conducted in the cloud, which means having additional computational and storage powers at a remote site away from the direct user. It is best utilized in certain scenarios like making market predictions, detecting attempts at fraudulence, warehouse management and a host of other things. Cloud-based machine learning produces different outputs based on what input is provided. It can help a business recognize the answer to a yes or no decision or other decision-based questions. This has real-world implications for when an order is made or when a video streaming platform decides what video to recommend to the user. Cloud-based machine learning also helps to categorize data that has been input by users. For instance, it may help a law firm assess whether a lawsuit filed for a claim is legitimate or not. It judges this based on the previous learning processes and is also capable of making additional decisions on its own without further data input.

Furthermore, cloud-based machine learning can make predictions based on past data that has been put into the system. The predictions are very accurate, and businesses may depend upon them to make important decisions. One of the scenarios where cloud-based machine learning may be useful is to assist in supply chain decisions. A business may input past data into the system to forecast exactly how much inventory they may or may not need at a store outlet during a certain period of time. The possible uses of cloud-based machine learning are endless. As more technological advancements are, more uses are likely to be discovered.

Cloud-based machine learning has the potential to grow into something that may be highly beneficial to people and businesses. Until very recently, the majority of businesses had no access to resources that could enable them to reap the benefits of cloud-based machine learning. More recently, companies like Amazon, Google and Microsoft have brought their own cloud-based machine learning services to people. Each of these platforms shares common features, and each has a unique set of pros and cons. The tech giants provide their corporate customers with the required hardware to properly conduct machine learning in the cloud. And the hardware modules are specifically designed to handle machine learning in the cloud efficiently.

The most amazing thing about cloud-based machine learning is that the more it is done, the more it will develop with lesser human effort. Currently, the demand for cloud-based machine learning services is on the rise in the market. Prediction models, automated machine learning and other useful tools can help cloud-based machine learning thrive in the distant future.

1.2 The First Three Industrial Revolution

Before the 1760s, most of society was dependent on agriculture and farming. Nobody thought that steam would become a source of energy for numerous uses. The period from 1760 to 1830 is largely deemed as the time of the first industrial revolution. It was during this period that the first steam engine was invented, which revolutionized how people worked and how the world perceived energy. For the first time, people began to understand that there was more than a single source of energy that could be utilized. The era of industrialization had truly been set into motion with this one invention, and mechanization took hold of the people. New processes of coal mining were discovered to run the steam-powered engines and other mechanical pieces of equipment. Different processes like forging, use of metals and coal mining were products of the first industrial revolution and continue to contribute their benefits to this day. Everything from trains to ships started to use steam engines to move faster towards their destinations. These movements towards steam engines, saw agriculture take a back seat and gave rise to factories for production. Unfortunately, the factories during the first industrial revolution looked nothing like the ones of today. There were truckloads of workers with no skills and factories looking to pay them dimes on the dollar for their work. The conditions at these factories were abysmal, and the workload was immense. Considering the immense workload, the factories were, for the most part, highly dysfunctional and inefficient with a history of gruesome accidents. In the long run, these factories paved the way for the factories that have sprung up today and the modern cities that humankind has built. Although much of the first industrial revolution was isolated to Britain, it soon gained traction in Belgium with the help of some Englishmen who saw the bigger picture at that time.

Around the end of the 19th century came along the second industrial revolution. It was the time when new energy sources like gas, electricity, and oil started to be utilized. As a direct result, the use of the combustion engine skyrocketed. From the utilization of these resources, the steel industries also saw a surge in their production. This period saw the first time when chemical synthesis was used to develop things like plastic, dyes, fibers, etc. All this development meant that the factories had also developed from the time of the first industrial revolution. Mass production, or assembly line production as it is known today, first started during the second industrial revolution. The Ford Motor Company proudly put its workers to work assembling the first assembly line cars known to mankind. Due to this pickup in production, there was a demand for more workers. A large number of people moved to live in cities than had never been seen at any time in the past. People were leaving behind their rural homes and coming to work in factories in the city. The second industrial revolution ushered in the baby steps of urbanization. As urbanization saw a steady rise, methods of communication also started to evolve. The telephone and telegraph were the boons of the second industrial revolution, and it changed the way the world communicated.

Further down the line around the early 1900s, the automobile changed how people moved from one place to another. Although the first commercial airplanes were not introduced until much later, the Wright brothers invented the first airplane during the earlier periods of the 1900s. It was during the second industrial revolution that gave rise to the economic and industrial structures revolving around mass production in the factories that had been set up.

The latest of the three, the third industrial revolution, brought about drastic changes for mankind. Nuclear energy was discovered as an energy source, and a spike was seen in the production of electronics. This was caused by the invention of the transistor in 1947. The transistor also led to the invention of the microprocessor in the early 1970s. All these new technologies made it so that things could be manufactured in miniature form, in contrast to their previous bulky forms. This opened the gates to space research, and humans took their first steps on the moon in 1969 onboard the Apollo 11. For the growing industries, the introduction of transistors meant that many previously manual processes could now be automated. This meant there would less of a human effort in production. As production limitations were now lesser, mankind faced a boom in growth. Within no time, semiconductors, computers, and the then worldwide web had started a phenomenon like no other. All analog technologies were migrating to digital, and the world was increasingly becoming a place where small devices were preferred. These disruptive technologies changed entire industries and laid the pathway for the wonders of modern technology that are experienced today.

Most importantly, computers and computing power grew exponentially, which gave way to newer technologies. Newer ways of production were also unearthed, which greatly decreased the margin of error and helped boost efficiency levels. Supply chains