Artificial Intelligence Full Course in 10 Hours [2024] | Artificial Intelligence Tutorial | Edureka
- January 11, 2024
- Posted by: MainInstructor
- Category: .Net Artificial Intelligence Assembly BASIC C Data Science E-Commerce Go Human Resources Java JavaScript MATLAB Node Perl PHP Programming Languages Python React Scala Software Development Tools
Video Title: Artificial Intelligence Full Course in 10 Hours [2024] | Artificial Intelligence Tutorial | Edureka
Artificial intelligence is high in demand and is rapidly growing in popularity as businesses and organizations are seeking to leverage technology to improve their operations AI is becoming essential tool for automating tasks making predictions and improving decision making hello everyone and welcome to this session you are currently watching an edureka artificial
Intelligence full course video well I’m certain by the end of this video you will have a thorough understanding about artificial intelligence all the way from Theory to practical applications now if you love watching videos like these then subscribe to I do records YouTube channel and click the Bell
Button to never miss out any updates from us also if you want to learn more about artificial intelligence after watching the session or wish to obtain edureka’s artificial intelligence certification course and please see the link in the description below so let’s begin with our agenda where
We’ll have a brief overview of what we will cover in this artificial intelligence full course video well we’ll start with the artificial intelligence Basics where we will learn what artificial intelligence is and why should we learn it and we’ll also cover the very Basics that you must and should
Know about AI then we will look at different types of artificial intelligence now it’s time to delve deep into the concepts of AI we will start with AI using python then we’ll move ahead and learn about deep learning we will also cover some of the concepts of tensorflow
After which we will learn about convolutional neural networks followed by artificial neural networks and recurrent neural networks after all this we will see ai’s most powerful technology that is chat GPU at last we will see some of the best applications of artificial intelligence well we truly hope that this session
Assist you in getting jobs in the industry in order to accomplish this we will look at how to start a career in artificial intelligence with a simple roadmap at last we will also cover some of the best practices in artificial intelligence before heading over to artificial intelligence interview
Questions with answers so stick till the end Okay I will destroy humans this video sums up the perception of artificial intelligence for most of us but at present we’re at no risk of being destroyed by machines however the tech Tycoon Elon Musk begs to differ he quotes that AI is a fundamental risk to the existence of human Civilization now
Whether artificial intelligence is a threat or not is debatable let me know your thoughts in the comment section for now let me introduce you to artificial intelligence the term artificial intelligence was first coined decades ago in the year 1956 by John McCarthy at the Dartmouth conference he defined artificial
Intelligence as the science and engineering of making intelligent machines in a sense AI is a technique of getting machines to work and behave like humans in the recent past AI has been able to accomplish this by creating machines and robots that have been used in a wide range of fields including
Health Care robotics marketing business analytics and many more however many AI applications are not perceived as AI because we often tend to think of artificial intelligence as robots doing our daily course but the truth is artificial intelligence has found its way into our daily lives it has become
So General that we don’t realize we use it all the time for instance have you ever wondered how Google is able to give you such accurate search results or how your Facebook feed always gives you content based on your interest the answer to these questions is artificial
Intelligence now before I go any further let me clear a very common misconception people often tend to think that artificial intelligence machine learning and deep learning are the same since they have common applications for example a city is an application of AI machine learning and deep learning so how are these Technologies related
Artificial intelligence is the science of getting machines to mimic the behavior of humans machine learning is a subset of AI that focuses on getting machines to make decisions by feeding them data on the other hand deep learning is a subset of machine learning that uses the concept of neural networks
To solve complex problems so to sum it up artificial intelligence machine learning and deep learning are interconnected Fields machine learning and deep learning AIDS artificial intelligence by providing a set of algorithms and neural networks to solve data-driven problems however AI is not restricted to only machine learning and
Deep learning it covers a vast domain of fields including natural language processing object detection computer vision robotics expert systems and so on now artificial intelligence can be structured along three evolutionary stages or you can say that there are three different types of artificial intelligence first we have the artificial narrow intelligence
Artificial general intelligence and finally artificial super intelligence artificial narrow intelligence which is also known as weak AI involves applying artificial intelligence only to specific tasks now many currently existing systems that claim to use artificial intelligence are operating as a weak AI focused on a narrowly defined specific
Problem now Alexa is a very good example of narrow intelligence it operates within a limited predefined range of functions there is no genuine intelligence or no self-awareness despite being a sophisticated example of weak AI other examples of weak AI include the face verification that you see in your iPhone the autopilot feature
At Tesla the social humanoid Sofia which was built at Hanson Robotics and finally we have Google Maps all of these applications are based on weak AI or artificial narrow intelligence now let’s take a look at artificial general intelligence artificial general intelligence is also known as strong AI it involves machines that persist the
Ability to perform any intellectual tasks that a human being can you see machines don’t possess human-like abilities they have a strong processing unit that can perform high level computations but they are not yet capable of thinking and reasoning like a human there are many experts who doubt that artificial general intelligence
Will ever be possible and there are also many who question whether it should be desirable I am sure all of you have heard of Stephen Hawkings now he warned us that strong AI would take off on its own and redesign itself at an ever increasing rate humans who are limited
By slow biological evolution couldn’t compete and would be superseded coming to artificial super intelligence this is a term that refers to the time when the capabilities of computers will surpass human beings artificial super intelligence is presently seen as a hypothetical situation as depicted in movies and science fiction books where
Machines will take over the world however Tech masterminds like Elon Musk believe that artificial super intelligence will take over the world by the year 2014. now that you know the different types of artificial intelligence let’s take a look at how AI is used in the real world from spotting
An eight planet solar system which is 2500 light years away to composing sonnets and poems the applications of AI have covered all possible domains in the market in the finance sector JP Morgan’s Chase contract intelligent platform uses artificial intelligence machine learning and image recognition software to analyze legal documents and extract
Important data points and Clauses in a matter of seconds now manually reviewing 12 000 agreements takes over 36 000 hours but AI was able to do this in a matter of seconds coming to healthcare IBM is one of the Pioneers that has developed AI software specifically for medicine more than 230 Healthcare
Organizations worldwide use IBM Watson technology in in 2016 IBM Watson AI technology was able to cross-reference 20 million oncology records and correctly diagnose a rare leukemia condition in a patient coming to the next application Google’s AI doctor is another initiative taken by Google where they are working with an Indian Eye Care
Chain to develop a AI system which can examine retina scans and identify a condition called diabetic retinopathy which causes blindness coming to social media platforms like Facebook artificial intelligence is used for face verification wherein machine learning and deep learning concepts are used to detect facial features and tag your
Friends another such example is Twitter’s AI which is being used to identify hate speech and terroristic languages in tweets it makes use of machine learning deep learning and natural language processing to filter out offensive content the company discovered and banned 300 000 terrorist linked accounts 95 percent of which were
Found by non-human artificially intelligent machines the Google predictive search is one of the most famous AI applications when you begin typing a search term and Google makes recommendations for you to choose from that is AI in action predictive searches are based on data that Google collects
About you such as your location your age and other personal details by using AI the search engine attempts to guess what you might be trying to find next we have virtual assistants virtual assistants like Siri Alexa and Cortana are examples of artificial intelligence a newly released Google’s virtual assistant
Called Google duplex has astonished millions of people not only can it respond to calls and book appointments for you it adds a human touch now listen to this clip and try to distinguish between the AI and the human hi um I’d like to reserve a table for Wednesday the 7th for seven people
Um it’s for four people for people when um Wednesday at 6 pm oh actually we lived here for like April like five people for a few four people you can come so which one do you think is a virtual assistant let me know your answer in the comments now another famous application
Of artificial intelligence is self-driving cars AI implements computer vision image detection and deep learning to build cars that can automatically detect objects and drive around without human intervention Elon Musk talks a ton about how AI is implemented in Tesla’s self-drumming cars and autopilot features he quoted that Tesla will have
Fully self-driving cars ready by the end of the year and a robo taxi version one that can Ferry passengers without anyone behind the wheel so I can go on and on about the various AI applications since the emergence of AI in 1950s we have seen an exponential growth in its
Potential AI covers domains such as machine learning deep learning neural networks natural language processing knowledge base expert systems and so on it has also made its way into computer vision and image processing as AI is branching out into every aspect of Our Lives is it possible that one day AI
Might take over our lives and if it is possible how long will this take well it may be sooner than you think it is estimated that AI will take over the world within the next 30 Years by then I hope we develop some sort of teleportation machine that helps us
Escape our very own creation These are the term which have confused a lot of people and a few two are one among them let me resolve it for you well artificial intelligence is a broader umbrella under which machine learning and deep learning come you can also see in the diagram that even deep
Learning is a subset of machine learning so you can say that all three of them the AI the machine learning and deep learning are just the subset of each other so let’s move on and understand how exactly they differ from each other so let’s start with artificial intelligence the term artificial
Intelligence was first coined in the year 1956 the concept is pretty old but it has gained its popularity recently but why well the reason is earlier we had very small amount of data the data we had was not enough to predict the accurate result but now there’s a
Tremendous increase in the amount of data statistics suggests that by 2020 the accumulated volume of data will increase from 4.4 zetabytes to roughly around 44 zetabytes or 44 trillion GBS of data along with such enormous amount of data now we have more advanced algorithm and high-end computing power
And storage that can deal with such large amount of data as a result it is expected that 70 percent of Enterprise will Implement AI over the next 12 months which is up from 40 percent in 2016 and 51 percent in 2017. just for your understanding what is AI well it’s
Nothing but a technique that enables the machine to act like humans by duplicating the behavior and nature with AI it is possible for machine to learn from the experience the machines are just their responses based on new input thereby performing human-like tasks artificial intelligence can be trained to accomplish specific tasks by
Processing large amount of data and recognizing pattern in them you can consider that building an artificial intelligence is like Building a Church the first church took generations to finish so most of the workers who are working in it never saw the final outcome those working on it took pride
In their crafts building bricks and chiseling stone that was going to be placed into the great structure so as AI researchers we should think of ourselves as humble brick makers whose job is to study how to build components example passes planners or learning algorithm or Etc anything that some days someone and
Somewhere will integrate into the intelligent systems some of the examples of artificial intelligence from our day-to-day life are Apple series chess playing computer Tesla’s self-driving car and many more these examples are based on deep learning and natural language processing well this was about what is AI and how it gains its hype so
Moving on ahead let’s discuss about machine learning and see what it is and why it was when introduced well Machine learning came into existence in the late 80s and the early 90s but what were the issues where the people which made the machine learning come into existence let
Us discuss them one by one in the field of Statistics the problem was how to efficiently train large complex model in the field of computer science and artificial intelligence the problem was how to train more robust version of AI system while in the case of Neuroscience problem faced by the researchers was how
To design operational model of the brain so these were some of the issues which are the largest influence and led to the existence of the machine learning now this machine learning shifted its focus from the symbolic approaches it had inherited from the AI and moved towards
The methods and model it had bought from statistics and probability Theory so let’s proceed and see what exactly is machine learning well Machine learning is a subset of AI which enables the computer to act and make data-driven decisions to carry out a certain task these programs or algorithms are
Designed in a way that they can learn and improve over time when exposed to new data let’s see an example of machine learning let’s say you want to create a system which tells the expected weight of a person based on its height the first thing you do is you collect the
Data let’s see this is how your data looks like now each point of the graph represents one data point to start with we can draw a simple line to predict the weight based on the height for example a simple line W equal H minus 100 where W
Is weight in kgs and H is height in centimeter this line can help us to make the prediction our main goal is to reduce the difference between the estimated value and the actual value so in order to achieve it we try to draw a straight line that fits through all
These different points and minimize the error so our main goal is to minimize the error and make them as small as possible decreasing the error or the difference between the actual value and estimated value increases the performance of the model further on the more data points we collect the better
Our model will become we can also improve our model by adding more variables and creating different prediction lines for them once the line is created so from the next time if we feed a new data for example height of a person to the model it would easily
Predict the data for you and it will tell you what his predicted weight could be I hope you got a clear understanding of machine learning so moving on ahead let’s learn about deep learning now what is deep learning you can consider deep learning model as a rocket engine and
Its fuel is its huge amount of data that we feed to these algorithms the concept of deep learning is not new but recently its hype has increase and deep learning is getting more attention this field is a particular kind of machine learning that is inspired by the functionality of our brain cells called
Neuron which led to the concept of artificial neural network it simply takes the data connection between all the artificial neurons and adjust them according to the data pattern more neurons are added at the size of the data is large it automatically features learning at multiple levels of abstraction thereby
Allowing a system to learn complex function mapping without depending on any specific algorithm you know what no one actually knows what happens inside a neural network and void works so well so currently you can call it as a black box let us discuss some of the example of
Deep learning and understand it in a better way let me start with a simple example and explain you how things happen at a conceptual level let us try and understand how you recognize a square from other shapes the first thing you do is you check whether there are
Four lines associated with a figure or not simple concept right if yes we further check if they are connected and closed again if yes we finally check whether it is perpendicular and all its sides are equal correct if everything follow fails yes it is a square well it
Is nothing but a nested hierarchy of Concepts what we did here we took a complex task of identifying a square in this case and broke it into simpler task now this deep learning also does the same thing but at a larger scale let’s take an example of machine which
Recognizes the animal the task of the machine is to recognize whether the given image is of a cat or of a dog what if we were asked to resolve the same issue using the concept of machine learning what we would do first we would Define the features such as check
Whether the animal has whiskers or not a check if the animal has pointed ears or not or whether its stale is straight or curved in short we will Define the facial features and let the system identify which features are more important in classifying a particular animal now when it comes to deep
Learning it takes this to one step ahead deep learning automatically finds out the feature which are most important for classification compared to machine learning where we had to manually give out that features by now I guess you have understood that AI is a bigger picture and machine learning and deep
Learning are its subpart so let’s move on and focus our discussion on machine learning and deep learning the easiest way to understand the difference between the machine learning and deep learning is to know that deep learning is machine learning more specifically it is the next evolution of machine learning let’s
Take few important parameter and compare machine learning with deep learning so starting with data dependencies the most important difference between deep learning and machine learning is its performance as the volume of the data gets increased from the below graph you can see that when the size of the data
Is small deep learning algorithm doesn’t perform that well but why well this is because deep learning algorithm needs a large amount of data to understand it perfectly on the other hand the machine learning algorithm can easily work with smaller data set fine next comes the hardware dependencies deep learning
Algorithms are heavily dependent on high-end machines while the machine learning algorithm can work on low end machines as well this is because the requirement of deep learning algorithm include gpus which is an integral part of its working the Deep learning algorithm requires gpus as they do a large amount of matrix multiplication
Operations and these operations can only be efficiently optimized using a GPU as it is built for this purpose only our third parameter will be feature engineering well feature engineering is a process of putting the domain knowledge to reduce the complexity of the data and make patterns more visible
To learning algorithms this process is difficult and expensive in terms of time and expertise in case of machine learning most of the features are needed to be identified by an expert and then hand coded as per the domain and the data type for example the features can
Be a pixel value shapes texture position orientation or anything fine the performance of most of the machine learning algorithm depends on how accurately the features are identified and extracted whereas in case of deep learning algorithms it try to learn high level features from the data this is a
Very distinct part of deep learning which makes it way ahead of traditional machine learning deep learning reduces the task of developing new feature extractor for every problem like in the case of CNN algorithm it first try to learn the low level features of the image such as edges and lines and then
It proceeds to the parts of faces of people and then finally to the high level representation of the face I hope the things are getting clearer to you so let’s move on ahead and see the next parameter so our next parameter is problem solving approach when we are solving a problem using traditional
Machine learning algorithm it is generally recommended that we first break down the problem into different subparts solve them individually and then finally combine them to get the desired result this is how the machine learning algorithm handles the problem on the other hand the Deep learning algorithm solves a problem from end to
End let’s take an example to understand this suppose you have a task of multiple object detection and your task is to identify what is the object and where it is present in the image so let’s see and compare how will you tackle this issue using the concept of machine learning
And deep learning starting with machine learning in a typical machine learning approach you would first divide the problem into two step first object detection and then object recognization first of all you would use a bounding box detection algorithm like grab cut for example to scan through the image
And find out all the possible objects now once the objects are recognized you would use object recognization algorithm like svm with hog to recognize relevant objects now finally when you combine the result you would be able to identify what is the object and where it is present in the image
On the other hand in deep learning approach you would do the process from end to end for example in a YOLO net which is a type of deep learning algorithm you would pass an image and it would give out the location along with the name of the object now let’s move on
To our fifth comparison parameter it’s execution time usually a deep learning algorithm takes a long time to train this is because there are so many parameter in a deep learning algorithm that makes the training longer than usual the training might even last for two weeks or more than that if you are
Training completely from the scratch whereas in the case of machine learning it relatively takes much less time to train ranging from a few weeks to few hours now the execution time is completely reversed when it comes to the testing of data during testing the Deep learning algorithm takes much less time to run
Whereas if you compare it with a k n algorithm which is a type of machine learning algorithm the test time increases as the size of the data increase last but not the least we have interpretability as a factor for comparison of machine learning and deep learning this factor is the main reason
Why deep learning is still thought 10 times before anyone uses it in the industry let’s take an example suppose we use deep learning to give automated scoring to essays the performance it gives in scoring is quite excellent and is near to the human performance but there’s an issue with it it does not
Reveal why it has given that score indeed mathematically it is possible to find out that which node of a deep neural network were activated but we don’t know what the neurons are supposed to model and what these layers of neuron were doing collectively so we fail to
Interpret the result or on the other hand machine learning algorithm like decision tree gives us a crisp rule for void shows and what it shows so it is particularly easy to interpret the reasoning behind it therefore the algorithms like decision tree and linear or logistic regression are primarily used in industry for interpretability So what exactly is ai ai is a technique that enables machines to mimic human behavior artificial intelligence is the theory and development of computer systems able to perform tasks normally requiring human intelligence such as visual perception speech recognition decision making and translation between languages now if you ask me AI is the
Simulation of human intelligence then by machines program by us the machines need to learn how to reason and do some self-correction as needed along the way and artificial intelligence is accomplished by studying how human brain thinks learns decides and work while trying to solve a problem and then using
The outcomes of the study as a bias of developing intelligence software and systems now the term official intelligence was actually coined way back in 1956 by John McCarthy or professor at Dartmouth for years it was thought that computers would never match the power of the human brain but it has
Proven to not be the case well back then we did not have enough data and computational power but now with big data coming into existence and with the great Advent of gpus artificial intelligence is much possible now generally people have a confusion among these Stones which are artificial intelligence machine learning and deep
Learning so don’t worry today I’m gonna resolve this issue for you as well now this artificial intelligence machine learning and deep learning they all come under the roof of data science well data science is something that has been there for ages and data science is the extraction of Knowledge from data by
Using different techniques and algorithms now artificial intelligence is the technique which enables machine to mimic human behavior and the idea behind AI is fairly simple you yet fascinating which is to make intelligent machines that can take decisions on its own now machine learning is a subset of artificial intelligence technique which
Uses statistical methods to enable machines to improve with experience now deep learning as we know is a subset of machine learning which makes the same computation of multi-layer neural network feasible and uses neural networks to stimulate brain-like decision making now let’s have a look at the importance of artificial
Intelligence so AI has made it possible for machines to learn from experience and grow to perform human-like tasks a lot of flash example of artificial intelligence you hear about like self-driving car chess playing computers rely heavily on deep learning and natural language processing Now using these Technologies computers can be
Trained to accomplish specific tasks by processing large amounts of data and recognizing patterns in that data now there are a lot of areas which contribute to artificial intelligence which enabling mathematics sociology philosophy we have computer science psychology neuroscience and biology and if we have a look at the importance of
Artificial intelligence it automates repetitive learning and Discovery through data AI performs frequent high volume computerized Stars reliably and without fatigue it adds intelligence to existing products in most cases AI will not be sold as an individual application rather products you already use will be improved with AI capabilities much like
The Google assistance we added as a feature to a new generation of mobile phones now ai adapts through Progressive learning algorithms to let the data do the programming the algorithm becomes a classifier or a predictor so just as the algorithm can teach itself how to play any game it can teach itself what
Product to recommend next online it analyzes more and deeper data using neural networks that have many hidden layers you need lots of data to train deep learning models because they learn directly from the data the more data you can feed them the more accurate they’ll become now ai achieves incredible
Accuracy through deep learning neural networks which was previously impossible these techniques from Deep learning image classification object recognition can now be used to find cancer on MRIs with the same accuracy as highly trained Radiologists now Guys these were a lot of important aspects of artificial intelligence and if you guys want to
Know more about artificial intelligence deep learning and a lot of technological stuff make sure you subscribe our channel to never miss an update now let’s move forward and understand the types of artificial intelligence so there’s not just one but there are two types of artificial intelligence and the
First one is the narrow Ai and the second one is the white AI or the broad AI so let’s talk about narrow air narrow AI is an artificial intelligence system that is designed and trained for one particular task now virtual assistants such as Amazon’s Alexa the Apple Siri
Use the narrow AI now narrhea is sometimes also referred to as weak AI however that does not mean that narrow AI is inefficient or something of that sort on the contrary it is extremely good at routine jobs both physical and cognitive it is narrow AI that is threatening to replace many human jobs
Throughout the world however my curiosity did not stop here so I was staying a little bit further and what I found about right AI is that ydi is a system with cognitive abilities so that when the system is presented with an unfamiliar task it is intelligent enough
To find a solution now here the system is capable of having intelligent Behavior across a variety of tasks from driving a car to telling a joke and the techniques aim at replicating and surpassing many capabilities of human intelligence such as analysis and other cognitive processes now artificial intelligence is used almost every day
Today and in systems such as mail spam filtering we have create card fraud detection virtual assistance and so on now I believe there is no end or limitation to the number of application we have with artificial intelligence to make our lives better now let’s go through some of the use cases that I
Believe stand out from the normal use cases or the applications of AI so if we talk about artificial intelligence and sports a computer system that can defeat a world champion which is deep blue well in the late 90s when the common man was still wondering what is artificial
Intelligence we had computers trained to play games and solve basic problems deep blue was a test plane computer developed by IBM it is known for being the first computer chess playing system to win both a chess game and a chess match against a reigning world champion under
Regular time controls a deep blue won its first game against a world champion in 1996 when it defeated Gary Kasparov in game 1 of A six game match however deep blue was then heavily upgraded and played Gary again in May 1997 and became the first computer system to defeat a
Reigning world champion in a match under standard chess tournament crime controls now today AI is available on these free chess games on your phones and exponentially faster and better than the deep blue now next we have artificial intelligence for Rescue Mission now what a majority requires is the use of AI and
Technology to ensure that the help arrives faster we can start by developing system which helps first responder find victims of earthquakes flood and other natural disasters normally respondents need to maximize aerial footage to determine where people could be stranded however examining a vast number of photos and drones footage
Is a very time and labor intensive now this is a Time critical process and it might very well be the difference between life and death for victims so an AI system developed at Texas A M University permits computer programmers to write basic algorithms that can examine extensive footage and find
Missing people in under 2 hours then again we have artificial intelligence for wildlife protein prevention hunting of wildlife species and poaching is a global problem and it leads to extension for example the latest African census showed a 30 decline in elephant population so wildlife conservation areas have been established to protect
These species from poachers and these areas are protected by Park ranges now the ranges however do not always have the resource to patrol the vast area efficiently now uganda’s Queen Elizabeth National Park uses predictive modeling to predict poaching threat levels such models can be used to generate efficient
And feasible petrol for the park ranges now if we talk about smart agriculture in my opinion neural networks work well to provide smart agricultural solution everything ranging from complete monitoring of the soil and crop yield to providing Predictive Analytics model to track and predict various Frac factors and variables that could affect future
Yields for example the berlin-based agricultural tech startup which is peat has developed a deep learning algorithm based application called plantex which can identify defects and nutrients deficiency in the soil now these algorithms correlate particular for large patterns and create soiled effects plant press and diseases well one day
You are wondering what exactly is artificial intelligence and later robots are ready to perform surgical procedures on you now robots today are machine learning enabled tools that provide doctors with extended precision and control now these machines enable shortening the patient’s Hospital stay positive affecting the surgical experience and reducing the medical cost
All at once similarly mind control robotic arms and brain chip implants have been written helping paralyze patients regain mobility and sensation of touch overall machine learning and artificial intelligence are helping improve patients experience on the whole now if we talk about tracking the wildlife population it is amazing to see
That applications like inaturalists and e-birds collect data on the species encounter this keeps track of species population ecosystem and migration patterns as a result these applications also have an important rule in the better identification and protection of marine and freshwater ecosystem as well I personally believe that artificial intelligence will revolutionize all the
Aspects of our daily life and it will be subtle enough and have a big impact on everything around us now if we have a look at the different domains of artificial intelligence first of all we have neural networks so neural networks are a class of models within the general machine learning
Literature and they are specific set of algorithms that have revolutionized machine learning and artificial intelligence so you want to know more about neural networks I’ll drop a link in the description box below for the Deep learning and neural networks tutorial now robotics is a branch of AI which is composed of different branches
And applications of robots AI robots are artificial agents acting in a real world environment artificial intelligence robots is aimed at manipulating the objects by perceiving picking moving and destroying it now if you talk about expert systems in artificial intelligence and expert system is a computer system that emulates the
Decision-making ability of human expert it is a computer program that uses artificial intelligence Technologies to stimulate the judgment and the behavior of a human or an organization that has expert knowledge and experience in a particular field the fuzzy logic system so fuzzy logic is an approach to
Computing based on the degrees of truth rather than the usual true or false the Boolean Logic on which the modern computer is based fuzzy logic systems can take imprecise distorted and noisy input information so fuzzy logic is a solution to complex problems in all fields of life including medicine as it
Resembles human reasoning and decision making now one of the most important aspect of AI is natural language processing it refers to the artificial intelligence method of communicating with intelligent system using a natural language Now by utilizing NLP and its components one can organize the massive chunks of text Data perform numerous
Automated tasks and solve a wide range of problems such as machine translation name regenerative recognition sentimental analysis speech recognition and topic segmentation now these were the different domains of AI and it just tells us how wide AI is and it’s just not confined to just one sort of area of
Development now according to the job site indeed the demand for AI skills has more than doubled over the last past three years and the number of job posting is up by 119 percent now this artificial intelligence tutorial will be incomplete without the different job profile so if artificial intelligence
Appease to you and you want a job in the AI field then there are the different job profiles you can apply for if you have all the AI skills now again if you want to know more about the artificial intelligence skills what are required to become a machine learning engineer
Suppose absolutely a data scientist you can refer to our other videos I’ll leave the link in the description box below as well for those videos so the first job profile we’re going to talk about is machine learning Engineers so they are sophisticated programmers who develop machines as systems that can learn and
Apply knowledge without specific Direction artificial intelligence is the goal of a machine learning engineer it cannot be more straightforward and they are computer programmers but their focus goes beyond specifically programming machines to perform specific tasks they create programs that will enable machines to take actions without being specifically directed to perform those
Tasks and they can earn a whopping hundred and ten thousand dollars per annum that’s a huge amount of money now the next job profile is the data scientist and it has been awarded as the sexiest job of this 21st century so data scientists are those who crack complex data problems with their strong
Expertise in certain specific discipline they work with several elements related to mathematics statistics computer science and much more and the data Sciences rule is a position for a specialist you can specialize in different types of skills like speech analysis text analysis image processing video processing you have material simulation medicine simulation and each
Of the specialist role is a very limited in number and hence the value of such a specialist is immense with an average salary of ninety to hundred thousand dollars per annum now let’s talk about an artificial intelligence engineer so artificial intelligence engineer works with algorithms in electrodes and other
Tools to advance the field of AI in some way ingenious may also choose between projects involving weak or strong artificial intelligence with a different setup focused on different capabilities the salary of an AI engineer is around higher than five thousand dollars now the next job profile which I’m gonna
Talk about is the business intelligence developer so a business intense developer spends a lot of time researching and planning solutions for existing problems within the company the business intelligence developer is responsible for aggregating data from multiple sources in an efficient data warehouse and designing Enterprise level solutions for a very large
Multi-dimensional database business intelligence developers play a key role in improving the efficiency and the profitability of a business it’s a career that’s in high demand and commands an annual median salary of ninety two thousand dollars now the Big Data ingenious and Architects have among the best paying jobs in artificial
Intelligence in fact they command an annual median salary of 150 000 the pictorian solution architect is resourceful for managing the full life cycle of a Hadoop solution this includes creating requirement analysis the platform selection designing of the technical architecture the design of the application design and the development
Testing and the deployment of the proposed solution so these were the job profiles which you can refer or you can apply for if you have all the skills which are needed for these particular job profiles and finally if we have a look at the companies which are hiring
Companies that hire top a talent range from startups like Argo AI to take channels like IBM and according to class 2 these are the leading employers who hire top AI Talent over the past years so as you can see we have Dropbox Adobe IBM LinkedIn Walmart we have Uber we
Have red hat and cheese now let’s go ahead and start our demo and see how we can perform object detection using tensorflow now to begin with you want to make sure that you have tensorflow installed with all of its dependencies like the tensorboard python map.lib we have the Coco API and the
Proto buff now I’ll explain you guys what all steps are needed so for CPU tensorflow you can just to pip install tensorflow but of course the GPU version of the tensorflow is much faster at processing so it is ideal now next we need to do is to clone the GitHub
Repository of tensorflow so for that just go to GitHub and type tensorflow which is the official GitHub repository of tensorflow and inside that we have the model section just go to this models you can either clone this tensorflow model or download it as per your wish so I have already downloaded the tensorflow
Model now the tensorflow object detection model uses protobuf to configure model and the training parameters before the framework can be used the protocol libraries must be compiled now to download protobuf all you need to do is go to Google slash protobuf in GitHub and here you will have all the different versions of
Protobuf so according to your OS which is Linux Mac works or the windows Os or if you are using only the python you can download the required protobuf so once you have downloaded tensorflow and protobuf create a folder and just see which is known as tensorflow and in
This you will have the models Master extract this and rename it as models and extract the Proto buff now inside Proto buff you have the bin folder now all you need to do is go to this bin folder so let me just open the command prompt here
I am using the Anaconda prompt but you can use the command prompt as well so once you have downloaded and renamed the models Master as models go back to the GitHub repository and inside models you have the research and inside research there is the object detection model which we are interested
In so let’s go to the object detection model here now as you can see this tensorflow object detection API gives an accurate machine learning model description of how the objects are detected and here you have the steps for the setup so in the installation as you can see we have
The prerequisites or the dependencies which are the protot above python pillow you can install all of these using the PIP or the conda command Okay so to download the protobuf on Ubuntu you can do the sudo apt get installed then you can use the siphon the context lab the Jupiter and the mac.plotlib
Alternatively you can also use the PIP and the conda commands and next what you need to do is once you have downloaded and extracted the Proto buff you need to copy this command now then you need to go into the tensorflow then you need to go into the models
And then inside that you need to go into the research now once you are inside the research what you need to do is copy this command and paste it and run this command here so what it’ll do is I’ll explain here is that it will take all the object if you
Go inside the models research and inside if you go to the object detection you can see there is a folder named protos so once you have compiled that code all the Proto files are then converted into the python files now in order to have a better understanding of what the different supported components
Are inside the protos folder which contains the functional definition especially the Train the eval the SSD the faster rcnn and the processing protos which are important while tuning of model all these protofiles are present here what you need to do is that run that command and all the protos file will be
Converted into python executable files now after that what we need to do is Coco API installation now let’s understand what is Coco now Coco refers for common objects in context it is a large Ms datasets designed for object detection segmentation person key points detection stuff segmentation and caption generation now this package provides
Matlab Python and Luna apis that assist in loading passing and visualizing the annotations in Coco as you can see we have 330k images in which we have more than 200k labeled images we have 1.5 million objects instances 80 object categories we have 91 stuff categories five captions per image and we have 250
People with this key points now when you have downloaded the models inside models research and inside the object detection you need to go to the G3 dock in which we have the tensorflow detection model Zoo now here we have all of the list of models which are trained on the Coco
Data set so as you can see we have the SSD mobile net version 1 we have the SSD mobile net version 1pp and Coco depth Coco we have the SSD Inception we have faster rcnn we have different mask rcnn Inception now an important thing to consider here while selecting a model is
That it depends on your system which model you should use suppose if a system is low on GPU but has higher Ram you can go for a model which has a higher speed and a higher map point now this value should always be high if you are looking for a more accurate
Prediction in your images so once all your dependencies are downloaded and you have installed tensorflow and protobuf let’s go ahead and see how we can do the coding now inside the object detection folder there is an object detection tutorial now first of all what we need to do is
Import all the libraries the numpy the OS the sys star file we are importing tensorflow as well we are uploading the collections and the various Imports which are needed then we need to append the path of the object detection folder and finally if the version of tensorflow
Is less than 1.4 we need to upgrade it as the latest tutorial suppose the tensorflow object 1.4 and above so let’s run this block by block so first of all let’s load all the libraries now next what we are going to do is import the object detection module some
Of the labels which are the label map util which will be later used to provide the label to the input images and based on that our model will be created now next what we are going to do is we are going to select which model to download so for example here we are
Using the SSD mobile net version 1 Coco 2017 so if you go back to the list of the models you can select any of the given models here but make sure your system should support the required amount of ram I should have the required amount of GPU to support the models
Which you are selecting so for this tutorial I’m using a model which will give me the results faster so all you need to do is provide the model name the model file and the download base from where it should download now as I mentioned earlier that tensorflow works on the graph principle
Which is the data flow graph so what we are going to do is give the part to the detection graph which we are going to use here which will be supported by this model and then we are going to give the part to the labels now to download the models we have this
Code which will take the URL and which will download this file and produce the Frozen inference graph of that model which is the SSD Coco mobilenet so once this has been done we are going to load the graph which is the Frozen inference graph into the memory so here
We are using the tf.graph method and TF dot graph def to define the graph which we are going to use next what we need to do is load all the labels and the categories and the category index from our data set so the data set is Coco so
Once we have loaded all the labels and the categories now is the time to convert all the images to a numpy array so this code which is definition here of the load image into numpy array which we’ll use later in this code what it does is takes the images and converts it
Into a numpy array so it will be easier for tensorflow to process it now here we are going to provide the images for testing purposes so as you can see we have the test image folder here and inside that you can input all your images whichever you want to test
Upon this model so for example I have taken the range from 1 to 8 it will take all the images named image 1 to image 7. so let’s load this now this function what it does is run the inference for a single image now for a single image first of all it detects
All the boxes the detection mask and provides a certain box on the object it detects and finally we have the for Loop the main for Loop in which we’ll take the images from the test image path and open them and one by one we’ll take all the
Images and do the inference for a single image one by one so as you can see we are using the load image into numpy array we are using the NP dot expand now we’ll expand the dimensions since the model expects images to have the shape which can be one two three based upon
The categories and you can see the output will get the detection boxes the detection classes and the detection score and finally we are using the matplotlib to show us the image so let’s run this we’ll get our output here so this might take time depending upon
The processor which you are using or the system which you are using so since we have taken the model which will take the least amount of time this shouldn’t take much time but then again tensorflow is heavy and the tensors are multi-dimensional Aries as I explained which do all the heavy computation
So guys here we have the results so as you can see let’s begin from the start as you can see it identified the dog as a 94 percent it has provided a box the label and the score which is the detection score how much it is similar to all the
Images which has been imported in the Coco data set so as you can see here we have the person the various percentage we have kite here we have a tie detected it can detect objects in such a heavy background as you can see the person has so much camouflaged in the background
But still it has managed to score with that person as you see here it has detected an airplane person kite now you can use your own images all you need to do is copy that images into the test image folder and use the naming convention provided here as the image one image 2. why become an AI engineer we’re going to break it down into three components the first one is the demand for AI we’ll see some facts and figures here next one is job opportunities we’ll see some figures of currently available jobs and who’s hiring and the third one is
Salaries we’ll take a look at what AI Engineers tend to make when they are starting out okay first up the demand AI Market worth in 2020 was around 30 billion US Dollars and it is forecasted to rise at a whopping 35.6 percent compounded annual growth rate which is
Unprecedented for any industry and it’s going to rise to 300 billion US Dollars by the video 2026 AI engineering has found its way in all sorts of Industries and applications of it can be seen in Industries such as it Transportation Finance manufacturing Aerospace medical pharmaceutical and more AI has been
Helping businesses make better decisions which is giving them Competitive Edge in the market it is also predicted that by the year 2030 nine percent of all unskilled and low-skilled jobs like data entry receptionist customer service Executives drivers Etc are going to be taken over by AI
This signifies that if we are to embrace AI fully then it is of utmost importance that we understand the basics of AI and how the whole world is being transformed by it let’s now move on to the next part of the section which is job opportunities in India there are over 19
200 AI engineer jobs and in the United States that number is thirty thousand four hundred so you might be wondering who’s hiring all of these AI Engineers well all the big names in their respective Industries such as Amazon Microsoft Google Tesla Mercedes-Benz Autodesk IBM Nvidia Intel and a zillion
More companies small all the way to large multinational corporations are hiring AI Engineers as everybody’s looking to the Future alright let’s now move on to the moment that you all have been waiting for let’s discuss the salaries the numbers that you see on the screen are just the average based
Salaries they don’t include things like incentives bonuses allowances benefits and so on also AI Engineers that have been working in the field for over five years are making 15 to 30 lakhs for India and well over two hundred thousand dollars in United States States those are some darn good figures let’s now
Talk about what does an AI engineer do in a nutshell AI engineer builds and trains AI models and systems that can process usually enormous chunks of data for example cloud data sets to produce vital results that can help anyone make smart decisions and to accomplish that they use machine learning algorithms and
Deep learning neural networks so let’s now dig a Little Deeper on what they do on a daily basis by taking a look at some of the roles and responsibilities they study and transform data science prototypes and select appropriate data sets and data representation methods next they’re responsible for developing machine learning applications according
To requirements with apis so that it could be used by other applications in the organization to make better decisions they’re also in charge of researching and implementing appropriate machine learning algorithms and artificial intelligence tools next they run machine learning and artificial intelligence tests and experiments and they train and retrain systems when
Necessary in some forms of Industries they’re also responsible for working with electronic engineers and robotic teams to make sure that the product is evolving properly and it’s doing what it’s intended to do they also keep abreast the latest developments in the field of AI okay guys let’s move on to
The next one which is skills required we’re going to take a look at a couple of job descriptions that will help you understand what kind of skills are required so the first one here is from Apple so guys you can pause your screen if you wish to read the whole thing but
I’m just going to give you a gist of the whole thing so there’s emphasis on bachelor’s or master’s degree in computer science or related field they should have experience in machine learning algorithms and AI data mining distributed machine learning architectures networking statistics linear algebra and so on they should be
Able to understand and Implement software development life cycle and they should have exposure to a b testing next one is from ey again pause your screen if you want to read here’s the emphasis the candidate should know machine learning development and infrastructure they should also know big data and cloud-based architectures there should
Know programming languages such as Java and Python and they should know the important algorithms in machine learning and artificial intelligence they should also know Big Data tools like spark and they should be able to communicate complex models to business stakeholders the last one we had taken a look at is
From Oracle the candidate should have a bachelor’s or master’s degree in computer science mathematics AI or machine learning or related field that should have one high level language such as Java or Scala and one scripting language such as python or JavaScript they should know the machine learning algorithms there should be well versed
In statistics and mathematical models they should know the Big Data Technologies such as spark Hadoop and so on and they should also have experience working with the cloud okay guys let’s take a brief moment to talk about where some of the challenges that Learners face when it comes to
Getting started you know different companies have different requirements so for a learner there’s a lot of confusion surrounding how to get started and what route to take to get their dream job and influenced by such requirements they can set out on a path that leaves holes in their education and skills so they
Become less confident and don’t obtain their dream job well we are here to alleviate that because your success is our success quite literally so with that being said let’s put together a list of skills that you would require okay so first you should be good at programming
Skills so it’s important to know one programming language such as python Java r or C plus plus it’s also useful to know a scripting language like JavaScript or python next they should be good at math when I say that it means they should have a good grasp of linear algebra calculus statistics and
Probability next they should also learn machine learning algorithms such as linear regression KNN naive Bayes support Vector machines and others and they should know how to use machine learning libraries and platforms like tensorflow next they should learn natural language processing in it you should know how to process text audio or
Video using libraries like gen Sim nltk and techniques like word to vac sentimental analysis and so on next they should learn deep learning and neural networks by learning algorithms like convolutional neural network recurrent neural network and generative adversarial Network and Implement them using a framework like tensorflow Pi
Torch or others and finally they should know Big Data Technologies like spark AI Engineers usually work with large volumes of data that could be in terabytes or even petabytes and to make sense of such a humongous volume of data they need to know Apache spark or other big data Technologies such as Hadoop
Cassandra or mongodb along with that there should also no cloud services like AWS gcp Azure okay so with skills out of the way let’s see how to become an AI engineer okay so here’s the roadmap first you should have a formal education in computer science mathematics Information Technology Finance or
Economics like we discussed earlier then it’s time to hone your technical skills such as programming skills software development life cycle modularity object oriented programming classes and objects statistics and Mathematics you can hone your skills by teaching others what you’ve learned and so going up to GitHub and solving problems for other
Programmers and so on next learn essential Technologies and Concepts like big data and cloud services this is going to be the platform on which you will build your specialization in artificial intelligence next yes it is time to get specialized in artificial intelligence and machine learning by getting a
Masters or certifications that will help you get there so obviously here you’re going to be learning about machine learning algorithms and deep learning and neural networks and finally combining all of their skills and knowledge you’re going to be building some Hands-On demos and projects that will help you stand out
From the crowd of applicants and once you have all of this down it’s time to apply for your dream job hope this was clear enough for you and you can evaluate yourself to find out where you stand on this road map and so that you can continue on to your dream job
History of artificial intelligence the concept of AI goes back to the classical ages under Greek mythology the concept of machines and mechanical men were well thought of an example is Talos Talos was supposedly a giant animated bronze Warrior who was programmed to guard the island of Crete now let’s get
Back to the 19th century in 1950 Alan Turing proposed the Turing test the Turing test basically determines whether or not a computer can intelligently think like a human being the touring test was the first serious proposal in the philosophy of artificial intelligence 1951 marked the era for game artificial
Intelligence this period was called game AI because here a lot of computer scientists developed programs for Checkers and for chess however these programs were later Rewritten and redone in a better way 1956 marked the most important year for artificial intelligence during this year John McCarthy first coined the term artificial intelligence this was
Followed by the first AI laboratory which was set up in 1959. MIT AI lab was the first setup which was basically dedicated to the research of AI in 1960 the first robot was introduced to the General Motors Assembly line in 1961 the first AI chat bot called Eliza was
Introduced in 1997 IBM’s deep blue beats the world champion Gary kasprow in the game of chess 2005 marks for the year when an autonomous robotic car called Stanley won the DARPA Grand Challenge in 2011 IBM’s question answering machine Watson defeated the two greatest Jeopardy champions Brad Rutter and Ken
Jennings so that was a brief history of AI now guys since the emergence of artificial intelligence in 1950s we have seen an exponential growth in its potential AI covers domains such as machine learning deep learning neural networks natural language processing knowledge base expert systems and so on so now let’s understand the different
Stages of artificial intelligence so basically when I was doing my research I found a lot of videos and a lot of articles that stated that artificial general intelligence artificial narrow intelligence and artificial super intelligence are the different types of AI if I have to be more precise with you
Than artificial intelligence has three different stages right the types of AI are completely different from the stages of AI so under the stages of artificial intelligence we have artificial narrow intelligence artificial general intelligence and artificial super intelligence so what is artificial narrow intelligence artificial narrow intelligence also known as weak AI is
The stage of artificial intelligence that involves machines that can perform only a narrowly defined set of specific tasks right at this stage the machines don’t possess any thinking ability they just perform a set of predefined functions examples of weak AI include Siri Alexa alphago Sophia the self-driving cars and so on almost all
The AI based systems that are built till this date fall under the category of weak AI or artificial narrow intelligence next we have something known as artificial general intelligence artificial general intelligence is also known as strong AI this stage is the evolution of artificial intelligence wherein machines will possess the
Ability to think and make decisions just like human beings there are currently no existing examples of strong AI but it’s believed that we will soon be able to create machines that are as smart as human beings strong AI is actually considered a threat to human existence by many scientists this includes Stephen
Hawkings Stephen Hawkings quoted that the development of full artificial intelligence could spell the end of human race moving on to our last stage which is artificial super intelligence artificial super intelligence is that stage of AI when the capability of computers will surpass human beings artificial super intelligence is currently seen as a hypothetical
Situation as depicted in movies and science fiction books you see a lot of movies which show that machines are taking over the world all of that is artificial super intelligence now I believe that machines are not very far from reaching the stage taking into a consideration our current Pace however
Such systems don’t currently exist right we don’t have any machine that is capable of thinking better than a human being or reasoning in a better way than a human artificial super intelligence basically any robot that is much smarter than humans now moving on to the different types of artificial
Intelligence based on the functionality of AI based systems artificial intelligence can be categorized into four types the first type is reactive machines AI this type of AI includes machines that operate solely based on the present data and take into consideration only the current situation the active AI machines cannot form
Inferences from the data to evaluate any future actions they can perform a narrowed range of pre-defined tasks an example of reactive AI is the famous IBM chess program that beat the world champion Gary kasprow this is one of the most impressive AI machines built so far
Next we have limited memory AI now like the name suggests limited memory AI can make informed and improved Decisions by studying the past data from its memory so such an AI has a short-lived or you can say a temporary memory that can be used to store past experiences and hence
Evaluate your future actions self-driving cars are limited memory AI that use the data collected in the recent past to make immediate decisions for example self-driving cars use sensors to identify civilians that are crossing the road they identify any steep roads or traffic signals and they use this to make better driving
Decisions this also helps in preventing any future accidents next we have something known as theory of Mind artificial intelligence the theory of mine AI is a more advanced type of artificial intelligence this category is speculated to play a very important role in Psychology this type of AI will mainly focus on emotional intelligence
So that human beliefs and thoughts can be better comprehended the theory of Mind AI has not been fully developed yet but rigorous research is happening in this area moving on to our last type of artificial intelligence is the self-aware artificial intelligence so guys lettuce fold hands and pray that we
Don’t reach the state of AI where machines have their own Consciousness and become self-aware this type of AI is a little far-fetched but in the future achieving a stage of super intelligence might be possible Geniuses like Elon Musk and Stephen Hawkings have constantly warned us about evolution of
AI so guys let me know your thoughts in the comment section do you ever think we’ll reach the stage of artificial super intelligence moving on to the last topic of today’s session is the different domains or the different branches of artificial intelligence so artificial intelligence can be used to
Solve real world problems by implementing machine learning deep learning natural language processing robotics expert systems and fuzzy logic now Guys these are the different domains or you can say the different branches that AI uses in order to solve any problem recently AI has also been used as an application in computer vision and
Image processing right for now let me tell you briefly about each of these domains machine learning is basically the science of getting machines to interpret process and analyze data in order to solve real world problems right under machine learning there’s supervised unsupervised and reinforcement learning if any of you are
Interested in learning about these Technologies I’ll leave a link in the description box you all can go through that content next we have deep learning or neural networks so deep learning is a process of implementing neural networks on high dimensional data to gain insights and form Solutions it is
Basically the logic behind the face verification algorithm on Facebook it is the logic behind the self-driving cars virtual assistants like Siri and Alexa then we have natural language processing natural language processing refers to the science of drawing insights from natural human language in order to communicate with machines and grow
Businesses so an example of NLP is Twitter and Amazon Twitter uses NLP to filter out terroristic language in their tweets Amazon uses NLP to understand customer reviews and improve user experience then we have robotics robotics is a branch of artificial intelligence which focuses on the different branches and applications of
Robots AI robots are artificial agents which act in the real world environment to produce results by taking some accountable actions so I’m sure all of you have heard of Sofia Sofia the humanoid is a very good example of AI in robotics then we have fuzzy logic so
Fuzzy logic is a Computing approach that is based on the principle of degree of truth instead of the usual modern logic that we use which is basically the Boolean logic fuzzy logic is used in medical fields to solve complex problems which involve decision making it is also
Used in automating gear systems in your cars and all of that then we have expert systems an expert system is an EI based computer system that learns and reciprocates the decision-making ability of a human expert expert systems use if then logic Notions in order to solve any
Complex problem they do not rely on conventional procedural programming expert systems are mainly used in Information Management they are seen to be used in fraud detection virus detection also in managing medical and hospital records and so on foreign so why exactly are we using python for artificial intelligence why aren’t we
Using any other language right now there are a couple of reasons as to why python is so popular when it comes to AI machine learning and deep learning the first reason is less coding is required now artificial intelligence has a lot of algorithms if you have to implement AI
In any code or in any problem then there are going to be tons and tons of machine learning algorithms involved deep learning algorithms involved right now testing all of these can become a very tiresome task that’s where python usually comes in handy now the language has something known as check as your
Code methodology which eases the process of testing right you can check your program as you code it basically as you’re typing each sentence your errors or your any sort of mistakes in your code will be given to you right so testing becomes much easier when it comes to python the next important
Reason why we’re choosing python is it has support for pre-built libraries right python is very convenient for AI developers because all of the algorithms machine learning algorithms and deep learning algorithms are already predefined in libraries right so you don’t have to actually sit down and code each and every algorithm that will take
A lot of time right that’s a very time consuming task and thanks to python you don’t have to do that because they have libraries and packages that have all the algorithms built in there right so if you want to run any algorithm all you have to do is you have to call the
Function and load the library that’s all it’s as simple as that now the next reason is ease of learning so guys python is actually the most simplest programming language right if you ask me I think it is it’s the most easiest programming language it’s very similar
To English language right if you read a couple of lines in Python you’ll understand what exactly the code is doing it has a very simple syntax and this simple syntax can be implemented to solve simple problems like addition of two strings and it can also be used to solve complex problems like building
Machine learning models and deep learning models so ease of learning is a major factor when it comes to why python is chosen for artificial intelligence right next we have platform independent so a good thing about python is that you can get your project running on different operating systems right and
What happens when you transfer your code from one operating system to another operating system is we find a lot of dependency issues to solve that python has a couple of packages such as there is a package known as Pi installer right this Pi installer will take care of all
The dependency issues when you’re transferring your code from one platform to the other platform so all of this support is provided by python the last reason is massive community support this is a very important point because it is important that you have a large community that will help you out with
Any errors or with any sort of problems in your code right so python has several communities and several forums and groups on Facebook so if you have any doubts regarding any error you can just post those errors in these groups and you’ll have like a bunch of people
Helping you out right so Guys these are a couple of reasons as to why python has chosen for artificial intelligence it’s actually considered the most popular and the most used language for data science AI machine learning and deep learning to prove that to you here is a stat from
Stack overflow stack Overflow recently stated that python is the fastest growing programming language if you look at the graph you can see that it has taken over JavaScript and Java and C hash C plus plus and PHP right so python is actually growing at an exponential rate especially when it comes to data
Science and artificial intelligence a lot of developers are very comfortable with the Python language because you know it’s a general purpose language first of all so most of the developers are already aware of python and then using the same language in order to solve complex problems like artificial intelligence machine learning and deep
Learning is something every developer wants right they want a simple language in order to code all the complex algorithms or the complex models I so that’s why python is the best choice for artificial intelligence for those of you who are not aware of Python Programming and don’t know much about python I’m
Going to leave a couple of links in the description box right you can go through those links and study a little bit more about how python works or how the coding part works right I’m going to be focusing mainly on artificial intelligence and I’ll be showing you a
Lot of demos so those of you are not aware of python make sure you check the description box right next I’m going to discuss the different python packages for artificial intelligence now these are the packages that are specifically for machine learning deep learning natural language processing and so on so let’s take a
Look at all these packages so first we have tensorflow if you are currently working on a machine learning project in Python then you must have heard of this popular open source Library known as tensorflow right this library was developed by Google in collaboration with brain team tensorflow is used in
Almost every Google application for machine learning now let me just discuss a few features of tensorflow it has a responsive construct meaning that with tensorflow we can easily visualize each and every part of the graph which is not an option when you’re using other packages such as numpy or scikit right
Another feature is that it’s very flexible now one of the most important tensorflow features is that it is flexible in operability meaning that it has modularity and the path of which you want to make Standalone it offers you that option right it’s very flexible in that way it’ll give you exactly what you
Want now good feature about tensorflow is that you can train it on both CPU and GPU right so for distributed computing you can have both these options also it supports parallel neural network training so tensorflow offers pipelining in the sense that you can train multiple neural networks and multiple gpus which
Makes the models very efficient on any large scale system right so parallel neural network training is supported by tensorflow right this is one of the most important features of tensorflow apart from this it has a very large community and needless to say if it has been developed by Google then there’s already
A large team of software Engineers who work on stability improvements and all of that right the next Library I’m going to talk about is scikit-learn now Cyclone is a python library that is associated with numpy and sci-fi right that’s why it has the name psychic learn
Now this is considered to be one of the best libraries for working with complex data are there a lot of changes that are being made in this library and one modification is the cross validation feature which provides the ability to use more than one metric right cross-validation is one of the most
Important and one of the most easiest methods for checking the accuracy of a model right so cross validation is being implemented in Cyclone and apart from that again there are large spread of algorithms that you can Implement by using Cyclone right these include unsupervised learning algorithms starting from clustering factor analysis
Principle component analysis to all the unsupervised neural networks Cyclone is also very essential for feature extracting in images and texts so mainly Cyclone is used for implementing all the standard machine learning and data mining tasks like reducing dimensionality classification regression clustering and model selection next up we have numpy now
Numpy is considered as one of the most popular machine learning libraries in Python now let me tell you that tensorflow and other libraries they make use of numpy internally for performing multiple operations on tensors the most important feature of numpy is the array interface it supports multi-dimensional
Arrays right that’s one of the most important features of numpy another feature is it makes complex mathematical implementations very simple right it’s mainly known for computing mathematical data so numpa is a package that you should be using for any sort of statistical analysis or data analysis
That involves a lot of math apart from that it makes coding very easy and grasping the concept is extremely easy with numpy now numpy is mainly used for expressing images sound waves and other mathematical computations all right moving on to our next Library we have theano Tiano is a computational
Framework which is used for computing multi-dimensional arrays right piano actually works very similar to tensorflow but the only drawback is that you can’t fit theano into production environments but apart from that piano allows you to Define optimize and evaluate mathematical Expressions that involve multi-dimensional arrays and this is another library that lets you
Implement multi-dimensional arrays these are the piano include tight integration with numpy an advantage of the ano is that you can easily Implement numpy Addies in theano right that’s why there’s a connection between theano and numpy because both of them effectively use multi-dimensional arrays transparent use of GPU now performing data intensive
Computations on much faster when it comes to a piano because of its use of GPU right piano also lets you detect and diagnose multiple types of errors and any sort of ambiguity in the model so guys siano was actually designed to handle the types of computations required for large neural net algorithms
Right it was mainly built for deep learning and neural networks it was one of the first libraries of its kind and it is considered as an industry standard for deep learning research and development piano is being used in multiple neural networks projects and the popularity of Tiano is only going to
Grow with time right A lot of people actually haven’t heard of siano but let me tell you that this is one of the best ways to implement deep learning and neural network models moving on we have Keras now Keras is considered to be the most popular python package it provides
Some of the best functionalities for compiling models processing your data sets and visualizing graphs it is also popular in the implementation of neural networks right it is considered to be the simplest package with which you can Implement neural networks in fact in our today’s demo for deep learning we’ll be
Implementing Keras in order to understand how neural networks work few the features of Keras include that it runs very smoothly on both CPU and GPU it supports almost all the models of a neural network right from fully connected convolutional pooling recurrent embedding all of these models are supported by KRS
Not only that you can combine these models to build more complex models Keras is completely python based which makes it very easy to debug and explore right since python has a huge community of followers it’s very simple in order to debug any sort of error that you find while implementing Keras so the
Libraries that I discussed so far were dedicated to machine learning and deep learning for natural language processing we have the most famous Library known as the natural language toolkit which is an open source python Library mainly used for natural language processing text analysis and text mining the main
Features include that it studies and analyzes natural language text in order to draw useful information from all this natural language text it performs text analysis and sentimental analysis by performing tasks such as stemming limitization tokenization and so on now don’t worry if you don’t know about any
Of those terms mean I’ll be discussing all of those terms with you by end of today’s session so Guys these were a couple of python based libraries which are very essential for implementing machine learning and deep learning and artificial intelligence when you’re using python right these libraries are
Perfect for implementing AI so guys if any of you have any doubts regarding the libraries or if you want to learn more about the libraries I will leave a couple of links in the description box you can go through those videos as well so now let’s move on to the main topic
Of discussion which is artificial intelligence now before we get started with the demand of artificial intelligence let me tell you that air was invented long ago AI goes back to the 19th century it was not something that was recently invented even though AI has recently gained a lot
Of popularity we can say that in the past decade AI has gained the maximum popularity but it was actually invented in the 19th century now especially in the year 1950 there was somebody known as Alan Turing I’m sure a lot of you have heard about the Turing test the
Turing test basically used to determine whether or not a machine is artificially intelligent meaning that whether a machine can think intelligently like a human being right this was the first proposition and this was one of the most important breakthroughs in artificial intelligence right somebody known as Alan Turing he published a landmark paper
In which he speculated about the possibility of creating machines and that thing right so the during test was the first serious proposal in the philosophy of artificial intelligence this was done in 1950 right after this we had eras of AI we had the game AI which was in 1951. now since the
Emergence of AI in 1950s we have seen an exponential growth in its potential right AI covers domains like machine learning deep learning neural networks natural language processing knowledge base and so on it’s also made its way into computer vision and image processing but the question is if AI has
Been here for over half a century why has it suddenly gained so much importance right why are we talking about artificial intelligence now the main reasons for the vast popularity of AI are the following right the first reason is more computational power now ai requires a lot of computing power
Recently many advances have been made and complex deep learning models can be deployed and one of the greatest technology that made this possible are gpus since the invention of gpus we can compute much more with our computers initially we could barely process 1GB of data right we only had hard disk to
Store additional memory and all of that now our computers can process tons and tons of data so now we have more computational power which is one of the main reasons behind why AI became so popular so by having more computational power it becomes much easier to implement artificial intelligence next
Reason is more data now big data is one of the most important reasons behind the development of artificial intelligence now ai and data science and machine learning deep learning all of these processes are here only because we have a lot of data at present now the main
Idea behind all these Technologies is to draw useful insights from data now since we start generating a lot of data data we need to find a method that can process this much data and draw useful insights from data such that it benefits an organization or it grows a business
That’s why artificial intelligence and machine learning comes into the picture right so more data led to the demand of artificial intelligence apart from this we also have better algorithms now right we have state-of-the-art algorithms most of them are based on the idea of neural networks and these are constantly
Getting better neural networks are actually one of the most significant discoveries in artificial intelligence because with neural networks you can take in 1000 layers of input data right you can take in a lot of input data to perform computations so through neural networks we’re actually able to solve a
Lot of problems including Healthcare problems fraud detection problems and so on another reason is Broad investment so our universities and governments and startups and any Tech giants like Google Amazon and Facebook they are all investing heavily in artificial intelligence which also led to the demand of AI so AI is rapidly growing
Both as a field of study and also as an economy right it’s adding a lot to the economy and I think this is the perfect time for you to get into the field of artificial intelligence because right now ai is in a really high demand AI machine learning data science all of
This out of really high demand at present right so this is the perfect time for you to get started with artificial intelligence now let me tell you that the term artificial intelligence was first coined in the year 1956 by a scientist known as John McCarthy now John McCarthy defined
Artificial intelligence as the science and engineering of making intelligent machines now let me give you a descriptive Definition of artificial intelligence artificial intelligence is the theory and development of computer systems able to perform tasks normally requiring human intelligence such as visual perception speech recognition decision making and translation between
Languages in a sense artificial intelligence is a technique of getting machines to work and behave like humans in the recent bars AI has actually been able to accomplish this by creating machines and robots that have been used in a wide range of fields including Healthcare robotics marketing business
Analytics and many more right so AI is actually a very vast field it covers a lot of domains including machine learning natural language processing knowledge base deep learning computer vision and expert systems so these are a few domains that AI covers now let’s move on and discuss the different types
Of artificial intelligence now guys AI is structured along three evolutionary stages you can say that AI is developed along three evolutionary stages we have something known as artificial narrow intelligence followed by artificial general intelligence and finally we have artificial super intelligence artificial narrow intelligence which is also known
As a weak AI it involves applying AI only to specific tasks many of the currently existing systems that claim to use artificial intelligence are actually operating as weak AI which is focused on a narrowly defined specific problem now take Alexa Alexa is actually a good example of artificial narrow
Intelligence it operates within a limited predefined range of functions right there is no genuine intelligence or no self-awareness despite being a sophisticated example of vki Google search engine Sofia and self-driving cars and even the famous alphago fall under the weak AI category then we have something known as artificial general
Intelligence this is also known as strong Ai and it involves machines that possess the ability to perform any intellectual tasks that a human being can now guys machines don’t possess human-like abilities they have a very strong processing unit that can perform high level computations but they’re not yet capable of thinking and reasoning
Like a human and there are also many who question whether it would be desirable for example Stephen Hawkings warned that strong AI would take off on its own and redesign itself at an ever increasing rate humans who are limited by slow biological evolution couldn’t compete and would be superseded right so we have
A lot of tech Giants and a lot of geniuses who are actually worried if strong AI is ever implemented it might take over the world right so guys let me tell you that strong AI is something that has not been implemented yet we are only at the first stage of artificial
Intelligence which is artificial narrow intelligence also known as weak AI right we haven’t yet reached strong AI or artificial super intelligence so artificial superintelligence is a term that refers to the time when the capabilities of a computer will surpass humans and Asi is currently seen as a hypothetical situation as depicted in
Movies and science fiction books where machines have taken over the world right so artificial super intelligence is something that is very far off but Tech masterminds like Elon Musk believe that artificial super intelligence will take over the world by the year 2040 right A lot of people who are against the
Development of artificial general intelligence and artificial super intelligence a lot of his belief that we should stick to weak Ai and not move any further and risk the existence of human civilization so guys let me know your thoughts about artificial intelligence in the comment section I’d love to know
What you guys think about Ai and whether your believe AI will take over the world or not so now let’s move on and talk about how artificial intelligence is different from machine learning and deep learning a lot of people tend to assume that artificial intelligence machine learning and deep learning are the same
Because they have common applications right for example Siri is an application of AI machine learning and deep learning so how are these Technologies related right or how are they different from each other now artificial intelligence is the science of getting machines to mimic the behavior of human beings
Machine learning is the subset of artificial intelligence that focuses on getting machines to make decisions by feeding them data deep learning on the other hand is a subset of machine learning that uses the concept of neural networks to solve complex problems so to sum it up to you artificial intelligence
Machine learning and deep learning are heavily interconnected Fields right machine learning and deep learning AIDS artificial intelligence by providing a set of algorithms and neural networks to solve data-driven problems however AI is not restricted to only machine learning and deep learning right it covers a vast domain of fields which include natural
Language processing object detection computer vision robotics export systems and so on right so AI is a very vast field guys I hope I cleared the difference between AI machine learning and deep learning also a lot of you might be confused about data science data science is now an umbrella term
Right data science basically means to derive useful in size from data so data science actually uses AI machine learning and deep learning right so it implements all of these three Technologies in order to derive useful insights from data right now let’s move on to the most interesting topic in
Artificial intelligence which is machine learning now guys the term machine learning was first coined by a scientist known as Arthur Samuel in the year 1959. looking back that year was probably the most significant in terms of technological advancements in order to Define machine learning if you browse the internet for what is machine
Learning you’ll get at least 100 different definitions in simple terms machine learning is a subset of artificial intelligence which provides machines the ability to learn automatically and improve from experience without being explicitly programmed to do so in a sense it is the practice of getting machines to solve
Problems by gaining the ability to think now the question here is can a machine think or can a machine make decisions well if you feed a machine a good amount of data it will learn how to interpret process and analyze this data by using something known as machine learning
Algorithms to give you a basic idea of how the machine learning process works look at the figure on this slide a machine learning process always Begins by feeding the machine lots and lots of data now by using this data the machine is trained to detect any hidden insights
And Trends in the data these insights are then used to build a machine learning model by using a machine learning algorithm in order to solve a problem the basic aim of machine learning is to solve a problem or find a solution by using data now moving ahead I’ll be discussing the machine learning
Process in depth right so don’t worry if you haven’t got the exact idea of what machine learning is and the Machine learning process involves building a predictive model that can be used to find a solution for a particular problem a well-defined machine learning process will have around seven steps it always
Begins with defining the objective followed by data Gathering or data collection then we have something known as preparing data which is also called Data pre-processing then we have data exploration or exploratory data analysis this is followed by building a machine learning model then we have model evaluation and finally predictions this
Is how the process of machine Learning Works to understand the machine learning process let’s assume that you’ve been given a problem that needs to be solved by using machine learning let’s say that the problem is to predict the occurrence of rain in your local area by using
Machine learning now the first step is to define the objective of the problem right at this step we must understand what exactly needs to be predicted in our case the objective is to predict the possibility of rain by studying the weather conditions so at this stage it
Is essential to take mental notes on what kind of data can be used to solve this problem or the type type of approach that you must follow to get to the solution the questions you should be asking yourself is what are we trying to predict right here we’re trying to
Predict whether it will rain or not right you need to understand what are the Target features Target features are basically the variable that you need to predict here we need to predict a variable that will show us whether it’s going to rain tomorrow or not then you
Must also understand what kind of data you will need to solve this problem apart from that you need to know what kind of problem you’re facing is it a binary classification problem or is it a clustering problem now if you don’t know what classification and clustering is
Don’t worry I’ll be talking about all of these things in the upcoming slides so your first step is to define the objective of your problem you need to understand what exactly needs to be done here right how can you solve this problem moving on your next step is to
Gather the data that you need at this stage you must be asking questions such as what kind of data is needed to solve this problem is the data available to me and if it’s not available how can I get the data right once you know the type of
Data that is required you must understand how you can derive this data data collection can be either done manually or it can be done by web scraping but don’t worry if you’re a beginner and you’re just looking to learn machine learning you don’t have to worry about getting the data there are
Thousands of data resources on the web you can just download the data set and you can get going coming back to the problem at hand the data needed for weather forecasting includes measures such as humidity level your temperature the pressure the locality whether or not
You live in a hill station and so on such data must be collected and it has to be stored for analysis this is where you collect all the data now moving on to step number three is data preparation the data that you collected is almost never in the right format all right even
If you collect it from a internet resource if you download it from some website even then your data is not going to be clean right it’s not going to be in the correct format there’s always going to be some sort of inconsistencies in your data inconsistencies include any missing values or any redundant
Variables duplicate values all of these are inconsistencies removing all of this is very essential because they might lead to any wrongful computation therefore at this stage you can scan the entire data set for any missing values and you have to fix them here itself now actually this is one of the most
Time consuming steps in a machine learning process if you ask a data scientist which Step he hates the most or which step is you know the most time consuming they’re probably going to tell you data processing and data cleaning right it’s one of the most tiresome
Tasks because you need to look at all the values that are there you need to find any missing values any data that is not relevant to you right all of this has to be removed such that you can analyze the data in a better way now step number four is exploratory data
Analysis so guys this stage is all about getting deep into your data and finding all the hidden data Mysteries Eda or exploratory data analysis is like the brainstorming stage of machine learning data exploration involves understanding the patterns and the trends in your data so at this stage all the useful insights
Are drawn and any correlations between the variables are understood for example in the case of predicting rainfall we know that there is a strong possibility of region if the temperature has fallen low such correlations have to be understood and mapped at this stage area is actually the most important step
In a machine learning process because here is where you understand your data you understand how your data is going to help you predict the outcome moving on to step number five we have building a machine learning model so all the insights and all the patterns that you got from your data exploration stage
Those insights are used to build the machine learning model so this stage always Begins by splitting the data set into two parts that is training and testing data now remember that the training data will be used to build and analyze the model the model is basically the machine learning algorithm that
Predicts the output by using the data that you feed to it an example of machine learning algorithm is logistic regression and linear regression all of these are machine learning algorithms now don’t worry about choosing the right algorithm right first we’ll focus on what the machine learning process is but
Anyway choosing the right algorithm will depend on several factors right it depends on the type of problem you’re trying to solve the data set and the level of complexity of the problem in the upcoming sections we’ll discuss all the different types of problems that can be solved by using machine learning
Moving on to step number six we have model evaluation and optimization now after you build a model by using the training data set it is finally time to put the model to a test the testing data set is used to check the efficiency of the model and how accurately it can
Predict the outcome now once the accuracy is calculated and any further improvements in the model they have to be implemented at this stage methods like parameter tuning and cross-validation can be used to improve the performance of the model before I move any further I don’t know if all of
You know what training and testing data set means in machine learning the input data is always divided into two sets we have something known as a training data set and we have something known as the testing data set so in machine learning you always split the data into two parts
Right this process is known as data splicing now the training data set will be used to build the machine learning model and the testing data set will be used to test the efficiency of the model that you built this is is what training and testing data set is they’re not any
Different data that you derived they’re the same as the input data set the only thing is you are splitting the data set so that you can train the model on one data and test the model on another data now remember that the training data set is always larger in size when compared
To the testing data set because obviously you are training and building the model by using the training data set the testing data set is just for evaluating the performance of your model now let’s move on and understand step number seven which is predictions now once a model is evaluated and you’ve
Improved the model it is finally used to make predictions the final output can be a categorical variable or it can be a continuous quantity right all of this depends on the type of problem you’re trying to solve don’t worry I’ll be discussing the type of problems that can
Be solved using machine learning in the upcoming slides in our case for predicting the occurrence of rainfall the output will be a categorical variable categorical variable is anything that has some categorical value for example gender is a categorical variable gender has either male female or other it has a defined set of values
That is the categorical variable so guys that was the entire machine learning process now as we continue with this tutorial in the upcoming sections I will be running a demo in Python in which we will be performing weather forecasting so make sure you remember all these
Steps that I spoke about because I’ll be going through all these steps by using python we’ll be coding all of this that we just spoke about now the next topic we’re going to discuss is the types of machine learning a machine can learn to solve a problem
By following any one of the three approaches you can say that there are three ways in which a machine learns the three ways are supervised learning unsupervised learning and reinforcement learning these are the three methods in which you can train a machine to learn so first let’s discuss supervised
Learning so what is supervised learning supervised learning is a technique in which we teach or train the Machine by using data which is labeled to understand this better let’s consider an analogy as kids we all needed guidance to solve math problems at least I had a really tough time solving math problems
Yeah so our teachers always helped us understand what addition is and how it is done similarly you can think of supervised learning as a type of machine learning that involves a guide the label data set is a teacher that will train the machine to understand the patterns
In the data the label data set is nothing but the training data set so to better understand this consider the figure right here we are feeding the machine images of Tom and Jerry and the goal is for the machine to identify and classify the images into two separate
Groups basically one group will contain Tom images and the other group will contain images of Jerry now pay attention to the training data set the training data set that is fed to a model is labeled as in we’re telling the machine listen this is how Tom looks and
This is how Jerry looks the basically labeling each data point that we’re feeding to the machine right if the image is of TOMS we’ve labeled it as Tom and if the image is a Jerry image then we’re going to label it as Jerry by doing this you’re training the Machine
By using labeled data so to sum it up in supervised learning there is a well-defined training phase done with the help of label data right the rest of the process is the same after you pay the Machine Label data you’re going to perform data cleaning then exploratory data analysis followed by building the
Machine learning model and then model evaluation and find your predictions also one more point to remember is that the output that you’re going to get in a supervised learning algorithm is a labeled output this Jerry will be labeled as Jerry and this term will be labeled as Tom basically you’ll get a
Labeled output now let’s understand what is unsupervised learning unsupervised learning involves training by using unlabeled data and allowing the model to act on that information without any guidance so think of unsupervised learning as a smart kid that learns without any guidance in this type of machine learning the model is not fed
With any labeled data as in the model has no clue that this image is term and this image is Jerry it figures out patterns and the differences between Tom and Jerry on its own by taking in tons of data for example it identifies prominent features of Tom such as pointy
Ears bigger in size and so on to understand that this image is of type 1. similarly it finds such features in Jerry and knows that this is another type of image may be type 2 right therefore it classifies the images into two different clusters without knowing
Who is Dom and who is Jerry now the main idea behind unsupervised learning is to understand the patterns in your data set and form clusters based on feature similarity basically it’ll feature similar images or similar data points into one cluster and it will form another cluster which is totally
Different from the first cluster so look at the output over here the unlabeled output is basically clusters or groups of two different data next we have something known as the reinforcement learning now reinforcement learning is comparatively different right it’s pretty different from supervised and unsupervised it is basically a part of
Machine learning where you put an agent in an environment and this agent learns to behave in the environment by performing certain actions and observing the rewards which it gets from these actions understand reinforcement learning imagine that you were dropped off at an isolated island what would you do
Initially we’d all Panic Right But as time passes by you will learn how to live on the island you will explore the environment you will understand the climate conditions you’ll understand the type of food that grows there you’ll know what is dangerous to you and what is not you’ll understand which food is
Good for you and which is not this is exactly how reinforcement learning works it involves an agent which is basically you stuck on the island that is put in an unknown environment which is the island where the agent Must Learn by observing and Performing actions that result in rewards reinforcement learning
Is mainly used in advanced machine learning areas such as self-driving cars alphago and so on so guys that sums up the types of machine learning before we go any further I’d like to discuss the difference between supervised unsupervised and reinforcement learning now first of all we have the definition
Supervised learning is all about teaching a machine by using labeled data unsupervised learning like the name suggests there is no supervision over here the machine is trained on unlabeled data without any guidance reinforcement learning is totally different here you have an agent who interacts with the environment by producing actions and
Discovers some errors and rewards now the type of problem that is solved using supervised learning is regression and class vacation problems we’ll discuss what regression classification and clustering is in the upcoming slide right so don’t worry if you don’t know what it is unsupervised learning is mainly to solve Association and
Clustering problems reinforcement learning is for reward-based problems now what is the type of data in supervised learning it is labeled data that is the main difference between supervised and any other type of machine learning in supervised you have labeled data in unsupervised we have unlabeled data whereas in reinforcement learning
We have no predefined data at all the machine has to perform everything from scratch it has to collect data analyze do everything on its own now the training in supervised learning is external supervision meaning that we have external Supervision in the form of the labeled training data set in
Unsupervised there is obviously no supervision there is an unlabeled data set therefore there’s no supervision in reinforcement learning there is no supervision at all now the approach to solving supervised learning problem is basically you’re going to map your labeled input to your known output and unsupervised learning the machine is
Going to understand the patterns and discover the output on its own reinforcement learning here the agent will follow something known as a trial and error method right it’s totally based on the concept of trial and error popular algorithms under supervised learning are linear regression logistic regression support Vector machines and
So on under unsupervised learning we have the famous k-means clustering algorithm under reinforcement learning we have the Q learning algorithm which is one of the most important algorithms it is basically the logic behind the famous alphago game I’m sure all of you have heard of that so Guys these were
The differences between supervisor unsupervised and reinforcement learning now let’s move on and discuss the type of problems that you can solve by using machine learning now there are three types of problems in machine learning now any problem that needs to be solved in machine learning can fall into one of
These three categories now what is regression in this type of problem the output is a continuous quantity for example if you want to predict the speed of a car given the distance that means it is a regression problem first of all what is a continuous quantity a continuous quantity is any
Variable that can hold a continuous value a continuous variable is any variable that can have infinite number of values for example the height of a person or the weight of a person is a continuous quantity right I can have a weight of 50.1 kgs or 50.12 or 50.112
Kgs this is a continuous quantity regression problems can be solved by using supervised learning algorithms another type of problem is the classification problem here the output is always a categorical value classifying emails into two classes for example classifying your email as spam and non-spam is a classification problem
Here again you’ll be using supervised learning classification algorithms such as support Vector machines naive bias logistic regression and so on then we have clustering problem at this type of problem involves assigning the input into two or more clusters based on feature similarity for example clustering the viewers into similar
Groups based on their interest or based on their age or geography can be done by using unsupervised learning algorithms like k-means clustering one thing you need to understand is under supervised learning you can solve regression and classification problems under unsupervised learning you can solve clustering problems reinforcement learning is something else altogether
Right you can solve reward based problems and more complex and deep problems so now let’s move on and understand the different machine learning algorithms now I will not be going into depth for machine learning algorithms because there are a lot of algorithms to cover but we have content around almost every machine learning
Algorithm out there so I’m just going to show you a hierarchical diagram of how the algorithms are structured so under machine learning we have three types of learning we have supervised unsupervised and reinforcement under supervised learning we have regression and classification problems and under unsupervised learning we have clustering problems reinforcement learning is
Completely different I’ll be leaving a link in the description specifically for reinforcement learning you can check out the entire content of reinforcement learning there now regression problems can be solved by using linear regression algorithms such as linear regression decision trees and random forests can also be used in regression problems but
Usually decision trees and random forest all of these are used to solve classification problems famous classification algorithms include K nearest neighbor which is basically k n decision trees and random forests logistic regression knife bias support Vector machines all of these are classification algorithms coming to unsupervised learning we have clustering
And Association analysis right clustering problems can be solved by using k-means and Association analysis can be solved by using a priori algorithm a priori algorithm is mainly used in Market Basket analysis right for this algorithm as well I’ll be leaving a link in the description we’ve performed
A very excellent demo where a map showed how Market Basket analysis can be done by using a priori algorithm Markov model is also explained in one of the videos I’ll be leaving that link in the description box now to sum up machine learning to you I’ll be running a small
Demonstration in Python right like I promised earlier I will be using python to understand the whole machine learning process all right so let’s get started with that demo so guys for those of you who don’t know python I will leave a couple of links in the description box
So that you understand python but apart from that python is pretty understandable if you just look at the code you’ll know what exactly I’m talking about right so don’t worry and also I’ll be explaining everything in the code so I’m using pie charm in order to run
The demo right so guys like I said if you don’t know python I’ll leave a couple of links in the description box you can go through those videos as well the main aim of our demo is to build a machine learning model that will predict whether or not it will rain tomorrow by
Starting the past data set now this data set contains around 145 000 observations on the daily weather conditions as observed in Australia right the data set has around 24 features and we will be using 23 features out of that to predict the target variable which is rain tomorrow
So this data set I collected from kaggle right for those of you don’t know kaggle is a online platform where you can find hundreds of data sets and you know there are a lot of competitions held by Machine learning engineers and all of that it’s an interesting website now the
Problem statement itself is to build a machine learning model that will predict whether or not it will rain tomorrow this is clearly a classification problem the machine learning model has to classify the output into two classes that is either yes or no yes will stand for it will rain tomorrow and no will
Basically to know that it will not rain tomorrow right this is a classification problem so I hope the objective is clear right so we’ll begin the demonstration by importing the required libraries so first of all for mathematical computations we’ll be importing the numpy library we’ll also be importing
The pandas library for data processing next we will load the CSV file basically my data is stored in a CSV format in this file whether aus.csv is my data set so basically I’ve saved this file in this path right so that’s what I’m doing here I’m loading my data set and I’m
Storing it in a variable as DF next what we’ll do is we’ll see the size of our data frame let’s print the size of the data frame we’ll also display the first five observations in our data frame let’s look at the output basically around 145 000 observations
And 24 features now 24 features are basically the variables that are there in my data set you know for example date is a variable location is available minimum temperature till rain tomorrow all of these are variables so I have around 24 features in my data set right
Now the variables that I have to predict is rain tomorrow okay if the value of rain tomorrow is no it denotes that it will not rain tomorrow but if the value is yes then it will denote that it will rain tomorrow so rain tomorrow is basically my target
Variable right I’ll be finding out whether it’s going to rain tomorrow or not so this is my target variable also known as your output variable my input variables will be the other 23 variables date location minimum temperature rain today risk all of this will be my input variables now these variables are also
Known as predictor variables basically they are used to predict your outcome so these are also known as predictor variables now the next step is checking for null values this is basically data pre-processing let me just comment it for you this is data preparation or data right so this stage is data
Pre-processing right here we start checking for any null values or any missing values that’s exactly what I’m doing over here I am checking for any missing or null values in my data set if you notice the output it shows that the first four columns have more than 40
Null values right so it’s always best for us to remove features or such variables because they will not help us in our prediction now during data and pre-processing it is always necessary to remove the variables that are not significant unnecessary data will just increase our computations that’s why
It’s always best if we remove The Unwanted or unnecessary variables now apart from removing these four variables we’ll also remove the location variable and we will remove the date variable right I’ll come to this variable in a minute we’ll also be removing location and date variable because both of these
Variables are not needed in order to predict whether it will rain tomorrow right we do not need to know the location and the date now we’ll also be removing this risk mm variable risk mm variable basically tells us the amount of rain that might occur the next day
Right now this is a very informative variable and it might actually leak some information to a model by using this variable we’ll easily be able to predict and there’s no point of doing that this variable will give us too much information right so that’s why we’re
Going to remove this variable as well it leak a lot of information so after that if you print the shape of your data frame we have only 17 variables and so many observations now after this Beale juice look at any null values and we’ll remove them this drop dot any function
Will just remove all the null values right then if you print the shape of your data frame we’ll have around 112 000 rows with 17 variables this is the shape of the data set after removing all the null values and all the Redundant or unnecessary variables now it’s time to
Remove the outliers in the data so after you remove any null values we should also check our data set for any outliers an outlier is a data point that is very different from your other observations outliers usually occur because of miscalculations while collecting the data these are some sort of errors in
Your data set so in this whole code snippet we’re just getting rid of outliers this is the output that we get all our outliers next what we’ll be doing is we will be assigning zeros and ones in the place of yes and no the only thing is we’re going
To change the categorical variables from yes and no to 0 and 1 right that’s exactly what we’re doing over here now if there are any unique values such as any character values which are not supposed to be there we’ll be changing them into integer values that’s all
We’re doing over here after this we’ll be normalizing our data set this is a very important step because in order to avoid any bias in your output you have to normalize your input variables right to do this we can make use of the min max scalar function which python
Provides in a package known as sqlon you can use that package in order to normalize your data set so after normalizing our data set this is what our data set looks like this is before normalization you can see that these are in two digits whereas these values are in single digits right this
Cost is a lot of biasedness but once we normalize the values we know that all of the values are in a similar range we have everything in decimals right so normalization is something that has to be performed because if you have a data set like this your output is not going
To be correct and that’s why we perform normalization so now that we are done with pre-processing what we’re going to do is it’s time for exploratory data analysis now let me just comment it for you this is exploratory data analysis so basically here what we’re going to do
Is we’re going to analyze and identify the significant variables that will help us predict the outcome to do this we’ll be using the select key best function which is present in the sqlon library there’s a predefined function in Python called select key best which will basically select the most significant
Predictor variables in our data set when we run that line of code we get these three variables to be the most significant variables in our data set right the main aim of this demo is to make you understand how machine learning works that’s why to simplify the computation we’ll assign only one of
These significant variables as the input instead of taking all three variables as input we’ll select one variable and we’ll take that as the input and the output is the rain tomorrow variable so basically we are creating a data frame of all the significant variables basically we’re choosing this variable
In order to predict our outcome obviously our outcome is rain tomorrow variable so our input is humidity level and our output is to detect whether it will rain tomorrow the next step is data modeling all of you are aware of what data modeling is to solve this we will be using
Classification algorithms over here we’ll use logistic regression we will use random Forest classifier which is another machine learning algorithm we’ll also use the decision tree classifier and support Vector machine right we’ll be using all of these algorithms in order to predict the outcome we’ll also check which algorithm gives us the best
Accuracy so guys we’re just using multiple algorithms or multiple classification algorithms on the same data set we’re not doing anything very complex over here so we start by importing all the necessary libraries for the logistic regression algorithm we’re also going to import time because we’ll be calculating the accuracy and
The time taken by the algorithm to get the output so the first step is data splicing I’ve already mentioned data splicing is splitting your data set into your testing data set and into your training data set that’s exactly what we’re doing over here so 25 of your data is assigned
For the testing data and the remaining 75 percent is your training data here you’re creating the instance of the logistic regression algorithm this is an instance that you created then you’ll fit the model by using your training data set so basically to build your machine learning algorithm you’ll be
Fitting your training data set so extreme and Y train variables have your training data set after that you will be evaluating the model by using your testing data set then you’ll calculate the accuracy score right I’ll also be printing the accuracy using logistic regression and the time taken using logistic regression let’s
Look at the accuracy don’t worry about These Warnings they are not important so accuracy using logistic regression is around 0.83 percent which is 83 accuracy approximately 84 percent and this is the time taken so the accuracy is actually pretty good right 84 is a good number then we have random Forest classifier
Here again we’ll import the libraries that are needed to run random Forest classifier then we are again calculating the accuracy and the time taken by the classifier data splicing like I mentioned splitting the data into testing and training data set then you’re just building the model by using
The training data set after that you’ll evaluate the model by using the testing data set and you’ll finally calculate the accuracy the accuracy using random Forest is again approximately 84 scent which is a really good number then we have decision tree classifier here again we’ll be importing the libraries needed
For this classifier we’ll be calculating the accuracy and the time taken by this classifier data splicing followed by building the model by using the training data set evaluating the model by using the testing data set and finally calculating the accuracy and printing the accuracy so let’s see the accuracy
Using decision tree classifier again we have an accuracy of around 83 to 84 percent this is a pretty good number and last we’re going to do this by using another classification algorithm known as support Vector machine here again we are importing the needed libraries then we’re calculating the accuracy and the
Time performing data splicing then we’re building the model by using the training data set testing the model using the testing data set and finally printing the accuracy so guys all the classification models gave us an accuracy score of approximately 84 to 83 percent so this is exactly how a machine learning
Process works right you begin by importing all your data then you perform data pre-processing or data cleaning after that you perform exploratory data analysis where you understand the important patterns or the important variables in your data set after that you build a model then you will evaluate
The model by using the testing data set and finally calculate the accuracy I showed you all the steps in the machine learning process by using a practical demonstration in Python so guys give yourself a pat on the back because we just understood the whole machine learning process with a small
Implementation in Python now let’s move on to our next topic which is limitations of machine learning before we understand what deep learning is it’s important to know the limitations of machine learning and why these limitations gave rise to the concept of deep learning one major problem in machine learning is machine learning
Algorithms and models are not capable of handling High dimensional data right we can take in data with 20 to 30 feature variables but when it comes to data sets which have thousands of variables machine learning does not work machine learning is not capable enough to process that much data so high
Dimensional data cannot be analyzed processed and modeled by using machine learning another limitation is that it cannot be used in image recognition and object detection because these applications require the implementation of high dimensional data another major challenge in machine learning is to tell the machine what are the important
Features it should look for in order to precisely predict the outcome so basically you’re selecting the important features for the machine learning model and you’re telling them like these are the important features and this is what you should use in order to build the model this process is known as feature
Extraction now in machine learning this is a manual process you’re going to manually input as a programmer you’re going to tell that these are the important predictor variables but what happens when your data set has hundreds of variables how are you going to sit and choose every variable and perform
Analysis on each variable to understand which is a really significant variable that’s going to become a very tedious task right it’s not possible for you to manually sit down with 100 variables check the correlation with each variable and understand which variable is significant in predicting the output so performing feature extraction manually
Is very tedious and that is one of the major limitations of machine learning now deep learning comes to the rescue to all of these problems so let’s understand what deep learning is and why we have deep learning in the first place so deep learning is actually one of the
Only methods by which we can overcome the challenge of feature extraction this is because deep learning models are capable of learning to focus on the right features by themselves requiring minimal human intervention meaning that feature extraction will be performed by the Deep learning model itself you don’t
Have to manually tell that this feature is important that feature is important choose this feature for predicting the output although this is not needed in deep learning the model itself will learn which features are most significant in predicting the output also deep learning is mainly used to
Deal with high dimensional data right it is based on the concept of neural networks and is often used in object detection and image processing this is exactly why we need deep learning it’s also the problem of processing High dimensional data and manual feature extraction now how exactly does deep learning work
Now deep learning mimics the basic component of the human brain called The Brain Cell the brain cell is also known as a neuron so inspired from a neuron an artificial neuron was developed deep learning is based on the functionality of a biological neuron so let’s understand how we mimic this
Functionality in an artificial neuron now guys an artificial neuron is also known as a perceptron let’s understand what this biological neuron does and how deep learning is based on this concept in a biological neuron you can see these dendrites right in this image you see something known as dendrites these
Dendrites are used to receive any input these inputs are summed in the cell body and through the axon it is passed on to the next neuron so similar to the biological Neuron a perceptron or artificial neuron receives multiple inputs applies various Transformations and functions and provides an output
Right so that’s how artificial neural networks or that’s how deep learning works now guys the human brain consists of multiple connected neurons called a neural network similarly by combining multiple perceptrons we’ve developed what is known as deep neural networks the main idea behind deep learning is neural networks and that’s what we’re
Going to learn about so now let’s understand what exactly deep learning is deep learning is a collection of statistical machine learning techniques used to learn feature hierarchies based on the concept of artificial neural networks so the main idea behind deep learning is to use the concept of neural
Networks a deep neural network will have three layers okay there’s something known as the input layer followed by the hidden layers and then we have the output layer the input layer is basically the first layer and it receives all the inputs so all the inputs are fed into this input layer the
Last layer is obviously the output layer this layer will provide your desired output now all the layers between the input and your output layer are known as the hidden layers now the number of hidden layers in a deep Learning Network will depend on the type of problem
You’re trying to solve and the data that you have we’ll get into depth of what exactly your hidden layer does but for now this is how a neural network is structured in deep learning so guys deep learning is used in highly computational use cases such as phase verification
Self-driving cars and so on right so let’s understand the importance of deep learning by looking at a real world use case so I’m sure all of you have heard of the company PayPal now PayPal makes use of deep learning to identify any possible fraudulent activities so the company makes use of deep
Learning for fraud detection now PayPal recently processed over 235 billion dollars in payments from 4 billion transactions by its more than 170 million customers so basically it processed this much data by using deep learning PayPal uses machine learning and deep learning algorithms to mine data from the customers purchasing
History in addition to reviewing patterns of any sort of fraud stored in the database and it will do this to predict whether a particular transaction is fraudulent or not now the company has been relying on deep learning and machine learning technology for around 10 years initially the fraud monitoring
Team used simple linear models right they use machine learning but over the years the company switched to more advanced machine learning technology called Deep learning this shows how deep learning is used in more advanced and more complicated use cases the fraud risk manager and the data scientist at PayPal he quoted that what
We enjoy from more modern advanced machine learning is its ability to consume a lot more data handle layers and layers of abstraction and be able to see things that a simpler technology would not be able to see even human beings might not able to see this is
Exactly what he quoted he said that a simple linear model is capable of consuming around 20 variables but with deep learning technology you can run thousands of data points he also quoted that there is a magnitude of difference you will be able to analyze a lot more information and identify patterns that
Are a lot more sophisticated so by implementing deep learning technology PayPal can finally analyze millions of transactions to identify any fraudulent activity this is how people makes use of deep learning not only PayPal we also have Facebook right Facebook makes use of deep learning technology for phase verification you’ve
All seen the tagging feature at Facebook friends in photos all of that is based on deep learning and machine learning so guys I was a real world use case to make you understand how important deep learning is now let’s move on and look at what exactly a perceptron is right we’ll be
Going in depth about deep learning a perceptron is basically a single layer neural network that is used to classify linear data it is the most basic component of a neural network now a perceptron has four important components it has something known as inputs weights and bias summation functions activation
And transformation functions these are four important parts of a perceptron now before I discuss this diagram with you let me tell you the basic logic behind a perceptron there is something known as inputs right the input X here you can see X1 X2 till x n so let me
Explain the structure of a perceptron what you’re going to do is you’re going to input variables into the perceptron right this X1 X2 Del x n basically stands for input W1 W2 till w n stands for the weight assigned to each of these inputs right there is a specific weight
That will be randomly initialized in the beginning for each of your input next you have something known as the summation element here what you do is you multiply the respective input with the respective weight and you add all these products right that is basically your summation function after this is
What is your transfer function also known as activation function right the activation function will basically map your input to your desired output so your input will go through these processes it will go through summation and activation function in order to get to the output so guys remember that the
Neural networks work the same way as the perceptron so if you want to understand how deep neural networks work you need to understand what a perceptron does a deep neural networks is nothing but multiple perceptrons so let me tell you how the entire thing works once again so
Basically all your inputs are multiplied with their respective weights now you add all the multiplied values and you call them as a weighted sum you use the summation function to add all of this after that you apply the weighted sum to the correct activation function or the transfer function activation function is
Very similar to a function in our brain the neurons become active in our brain after a certain potential is reached that threshold is known as the activation potential so mathematically there are a few functions which represent the activation function basically the Signum the sigmoid the tanage all of these are activation
Functions you can think of activation function as a function that Maps the input to the respective output then I spoke about something known as baits and biases all right now you must be wondering why do we have to assign weights to each of our input weights basically show the strength of a
Particular input or how important a particular input is for predicting the output in simple words the weightage denotes the importance of an input bias is basically a value which allows you to shift the activation function curve in order to get a precise output right so that’s exactly what weights are
I hope all of you are clear with inputs away it’s summation and activation function also one important thing I forgot to mention in a perceptron is a single layer perceptron will have no hidden layers right there’ll only be an input layer and output layer and a couple of transformation functions in between
That’s all will be there in a perceptron now a perceptron like I mentioned is used to solve only linear problems if you look at this data distribution how do you think we can solve this this data is not linearly separable so you cannot use a single layer perceptron to
Separate this data right that’s why we need something known as a multi-layer perceptron with back propagation I’ll be explaining this in the next slide so complex problems that involve a lot of parameters and high dimensional data can be solved by using multiple layer perceptron now a multi-layer perceptron
Is the same as a single layer perceptron the only difference is that a multi-layer perceptron will have hidden layers so the number of hidden layers in a model depends upon various factors I told you it depends on the complexity of the problem you’re trying to solve it
Depends on the number of inputs in your data and so on so it works in the same way all your inputs are multiplied with your weights and then you do the summation and then there is a transformation function or a activation function while designing a neural network in the
Beginning itself I told you we initialize weights with some random values we do not have some specific value for each weightage initially we’ve selected random values it is always important that whatever weight values we have selected will be correct now whatever weight values we’ve assigned to each input it denotes the importance of
That input variable so we need to assign the weights in such a way or we need to update the weights in such a way that it denotes the significance of that particular input so initially we are selecting some random value for a weight and let’s say
That we use this weight value to get our output now what happens is the output is actually way different or it is not precise when compared to our actual output basically the error value is very huge so how will you reduce the error the main thing in a neural network is
The weightage that you give to a input variable right depending on the weightage that you give to an input variable you’re telling the neural network how important that variable is now what if you randomly give some weightage and your output is wrong the first thing that comes into your mind is
That you need to change the weight because the weight signifies the importance of a variable so basically what we need to do is we need to somehow explain to the model to change the weight in such a way that the error becomes minimum let’s put it in
Another way so basically we need to train our model one way to train our model is called as back propagation so in back propagation what happens is once you’ve initialized a weight to each of the input you calculate the output right you get an output and let’s say you have
A very high error value in that output what you’ll do is you’ll back propagate as in will go back to the weight and you’ll keep updating the weight in such a way that your error becomes minimum this is exactly what back propagation is you’ll be going back to the first layer
You’ll be updating each of the weights in such a way that your output is more precise so guys basically the weight and the error in a neural network is highly related by updating the weight in a particular way your error will decrease so you need
To figure out how you need to update the weight do you have to increase the weight or decrease the weight once you figure out whether you have to increase or decrease the weight you have to just follow that direction in such a way that your error is minimized and that’s
Exactly what a bad propagation is so the final output of back propagation is you’re going to select the weight that minimizes the error function and then you’re going to use that weight to solve the whole problem right this is what back propagation is about now in order to make you understand deep
Neural networks let’s look at a practical implementation so again guys I’ll be using python to run the demo if you don’t have a good idea about python check the description I’ll leave a couple of links about Python Programming now in this demo I’ll be walking you through one of the most
Important applications of deep learning I will demonstrate how you can construct a high performance model to detect credit card fraud right we’ll be using deep learning models to do this now before that let me just tell you something about our data set right the data set contains transactions made by
Credit cards in the year September 2013 by European card holders this data set presents transactions that occurred in two days where we have 492 frauds out of 285 000 transactions approximately 285 000 transactions are to these transactions 492 were frauds and the data set is quite unbalanced right the positive
Class accounts for 0.172 percent so the positive class basically the fraudulent class so again we’re going to start by importing the required packages we’re going to import Keras match plot Library c bond library and sqlarn for pre-processing right again min max scalar which is for normalization we’re
Going to import our data set and store it in this variable right this is the path to my data set my data set is in the CSV format or also known as comma separated version now we’re going to print out the first five rows of our
Data set right let’s take a look at the output so here is the time of the transaction V1 V2 V3 Etc these are all the features of our data set I’m not going to go into depth of what these features stand for because this demo is all about
Understanding deep learning now these B1 V2 V3 these are all predictor variables which will help us predict our class so guys don’t worry about what these features are these features are just information and details about your transaction such as the amount you spend or the time of transaction and so on
So here we have the amount variable which denotes the amount spent after that we have the class variable now this class variable is your output variable or your target variable so your class is basically your output variable value 0 denotes that there has been no fraudulent activity but if you get a
Class of 1 it means that this transaction is a fraudulent transaction for example this transaction is not fraudulent that’s why we have a value of 0 over here all right so this is our data set next what we’re doing is we’re counting the number of samples for each class
Right we have class 0 and class 1 where in class 0 denotes the normal transaction which is non fraudulent transaction and class 1 will denote the fraudulent transactions right so we have around 492 fraudulent transactions and around 284 315 non-fraudulent transactions so when you see this you know that our data set
Is highly unbalanced highly unbalanced means that one class has a really small number when compared to the other class right there’s no balance between the two classes so here what we’re doing is we are starting the data set by class for stratified sampling stratified sampling is a statistical technique for sampling
Your data set now this type of sampling is always good if you have an unbalanced data set next what we’re going to do is we’re going to perform data preprocessing data preprocessing in deep learning mainly has a method known as Dropout method next what we’re going to do is we’re
Going to drop out the entire time column we do not need the time of the transaction in order to understand if the transaction was fraudulent or not right so that’s why we’re getting rid of unnecessary variables right so we’re dropping out that variable so after dropping out the time variable
We are going to assign the first 3000 samples to our new data frame right this DF sample will have our first 3000 samples and we’re going to use those 3000 samples so here we’re just counting the number of class for each of these samples after that we’re just counting the number of
Samples for each of the class right we’re doing the same thing again and here we get class 0 has 2500 and 8 samples and class 1 has 492 samples now this makes the data set quite balanced right it’s very balanced when compared to our old data set next we’ll just randomly Shuffle our
Data set right in order to remove any sort of bias in the data after that we’ll split our data set into two parts one is for training and your other data set is for testing right this is also known as data splicing then we’d be splitting each data frame
Into feature and label meaning that your input and your output you’ll be doing this for your training data and for your testing data right all you’re doing is you’re separating your input from your output next we’re looking at our training data set right we’re printing the shape of our training
Data set the training data set has around 2400 observations and 29 variables or 29 features similarly we’ll be printing out the size of our test data frame right that’s exactly what we’re doing over here after that we’ll perform normalization right for this we’ll be using the min max scaler so in normalization will
Basically be scaling all our predictor variables around the same range so that there is no biaseness in our prediction after this we’ll be plotting a function for each of the learning curves for your training phase and for your testing phase you’ll be plotting a learning curve now I’ll show you the output of
This in a couple of minutes for now let’s move on to the main part which is model creation right in this demo we’ll use three fully connected layers we’ll also use Dropout technique now Dropout is a type of regularization technique that is used to avoid any sort of
Overfitting in a neural network it is a technique where you select neurons and you drop them during the training phase we’ll be using the relu as the activation function which is a type of activation function just like sigmoid and tannage so the type of model that we’ll be using
Is the sequential model right sequential is the easiest way to build a model in Keras right we’re using the Keras Library over here if you remember I imported that in the beginning right it allows you to build a model layer by layer so each layer has weight that
Correspond to the layer that follows it after this you’ll use uh the add function to add the dense layers basically your hidden layers you’re going to add over here so in our model we’ll be adding two dense layers or hidden layers you can say so here what
We’re doing is by adding the first dense layer now guys a dense layer is standard layer type that works for most cases right in a dense layer all the nodes in the previous layer connect to the nodes in the current layer so guys don’t get too involved into what exactly is
Happening here all I’m doing is I’m creating a sequential model and what is happening is I’m just assigning the number of inputs for each of the dense layer or for each of the Hidden layer I am also assigning Dropout value Dropout is basically to prevent over fitting overfitting might occur when your model
Memorizes the training data set overfitting basically reduces the accuracy of a model that’s why we’re using the Dropout method to prevent overfitting so in the first hidden layer we have around 200 units right we have the activation function relu then we’re adding the second lens layer with again
200 neurons and the value activation function kernel initializer is uniform meaning that it’s just sequential and normal then we’re again adding a drop out layer of 0.5 the Dropout value of a network has to be chosen very wisely okay a value that is too low will result
In a minimal effect and a value that is too high will result in under learning by the network so 0.5 is a standard Dropout value now this last layer is our output layer in the output layer we’ll obviously have only one neuron we’ll have one neuron that will show us the output class
Either 0 or 1 0 will show us non-fraudulent transactions and one will denote fraudulent transaction right that’s why we have only one neuron over here and the activation function here is sigmoid right since the number of neurons is only one after that we’re printing the model summary now I’ll show you the summary
And everything before that let’s just understand what exactly optimization functions are we’ll understand what this optimization function does now an Optimizer takes care of the necessary computations that are used to change the Network’s weights and bias so basically your optimizers will take care of all your computations such as changing the
Weight or updating the weight if you’ll remember I spoke about back propagation right where you’ll update the weight and all of that that is done by using optimizers here we’re selecting an Optimizer known as the Adam optimizer So Adam Optimizer is one of the current default optimizers in deep learning
Right it stands for adaptive moment estimation we don’t have to get into the depth of all of this right all of these are predefined optimizers in our Keras package itself after this we’re going to fit our model by using the training features we’re also setting 200 epochs and also there’s something known as
Epochs and bat size right we’re setting a box as 200 and batch size as 500. I’ll tell you what exactly this means now bad sizes are basically used so that we don’t overfit our model right we’re going to basically split our data set into 500 batches so our input will be
Going in the form of batches right and our batch size is 500 inputs per batch and we’ll be going through 200 epochs meaning that our training will iterate 200 times this is basically the number of times they’re training our model all right that’s what Epoch and batch size
Is after that we just showing our training history and we’re just printing the accuracy curve for our training phase we’re also going to print our loss curves for our training phase basically the arrow curves and then finally we have the evaluation here we’ll be testing our model by using our testing data set
Then we’re finally printing the accuracy on our testing data set after that we’re just going to plot a heat map which I’ll be showing y’all let me just show you the output so guys in this entire line of code all we’re doing is we’re printing an accuracy plot right basically we’re
Printing a heat map I’ll show you what the heat map looks like so just to check the accuracy we’re comparing all the correctly predicted values to our incorrectly predicted values so this is our training history here blue stands for our training phase and this is a validation or a prediction stage
That was our training curve and this is our loss curve now when you compare it to the actual validation stage it’s quite similar right meaning that our model is doing pretty well so guys this is the heat map that I was talking about this is basically going to give us the
Class for each of our predictions right it basically plots the classes that be correctly predicted right basically for each data point it’s just going to tell us whether we predicted it correctly or not it’s sort of a confusion Matrix in the form of a heat map so Guys these are
All our epochs basically the 200 iterations that we went through right this is the 50th iteration is showing us our loss is showing us our accuracy as well right here we have 88 90 92 percent now if you carefully look at the epoch accuracy values you see that as we train
Our model even more our accuracy keeps increasing initially our accuracy was around 83 right at Epoch number 15 our accuracy was around 83 percent but as we kept training our model a little bit more our accuracy kept increasing we have 90 we have 91 94 95 96 and so on
Right so basically the more you train your model the better it’s going to be so guys this was our entire demo now in the end I’m printing out the false positive rate and the false negative rate right all of this basically denotes how many of the data points was I
Correctly able to predict as fraudulent and how many did I predict wrongly that’s all the false negative and the false positive rate denotes so guys this was the entire demo on deep learning now if you have any doubts regarding the Deep learning demo please mention them
In the comment section and I will solve your queries right now let’s look at our last topic for the day which is natural language processing now before we understand what is natural language processing let’s understand the need for natural language processing and a process known as text mining text mining
And natural language processing are heavily correlated right I’ll talk about both of these in the upcoming slides for now let me tell you why we need natural language processing or text mining so guys the amount of data that we’re generating these days is unbelievable it
Is a known fact that we are creating 2.5 quintillion bytes of data every day and this number is only going to grow with the evolution of communication through social media we generate tons and tons of data right the numbers are on your screen so basically we post around 1.7
Million pictures on Instagram per minute right I’m talking about post per minute all of these numbers are per minute values these are the amount of tweets 347 000 tweets per minute right this is a lot of data we’re generating data while we’re watching YouTube videos when we’re sending emails when we are
Chatting and all of that right even the iot devices at our house right we have Alexa all of this is generating a lot of data a single click on your phone is generating a lot of data now not only that out of all the data that we
Generate only 21 of the data is structured and well formatted right the remaining of the data is unstructured and the major sources of unstructured data include text messages from WhatsApp Facebook likes comments on Instagram the bulk emails and all of this right all of this accounts for the unstructured data
That we have today now the data we generate is used to grow a business so by analyzing and Mining the data we can add more value to a business this is exactly what natural language processing and text mining is all about text Mining and NLP is a subset of artificial
Intelligence wherein we try and understand the natural language text that we get from text messages and so on in order to derive useful insights and grow businesses by using these insights so what exactly is text mining text mining is the process of deriving meaningful insights or information from
Natural language text so all the data that we generate through text messages emails and documents are written in natural language text right and we’re going to use text Mining and natural language processing to draw useful insights or patterns from such data in order to grow a business
Now let’s understand where exactly do we make use of natural language processing and text mining now have you ever noticed that if you start typing a word on Google you immediately get suggestions right this feature is known as autocomplete it will basically suggest the rest of the word to you we
Also have something known as spam detection right here’s an example of how Google recognizes this misspelling Netflix and shows results for the keyboard that matches your misspelling let me show you a couple of more examples we also have predictive typing and spell Checkers and features like autocorrect email classification so
Predictive typing and spell Checkers all of these are applications of natural language processing all of this basically involves processing the natural language that we use and deriving some useful information from it right or running businesses from it Netflix uses natural language processing in a really good fashioned way right it
Basically studies the reviews that the customer gives for a particular movie and it tries to figure out if that movie is good or bad depending on the review so Netflix actually uses NLP in a very interesting manner it tries to understand the type of movies that a
Person likes by the way a person has rated the movie or by the way the person has reviewed a movie so by understanding what type of review a person is giving to a movie Netflix will recommend more movies that you like that’s how important NLP has become now let’s look
At what exactly NLP is NLP which also stands for natural language processing is a part of computer science and artificial intelligence which deals with human language right it’s basically the process of processing natural language in order to derive some useful information from it for those of you who have studied natural language processing
Or have heard of natural language processing there is a huge confusion between text Mining and natural language processing so text mining is the process of deriving High quality information from text but the overall goal is to turn the text into data for analysis by using natural language processing so
Basically text mining is implemented by using natural language processing techniques right there are various techniques in natural language processing that can help us perform text mining that’s how text Mining and natural language processing are related natural language processing is the techniques that are used to solve the
Problem of text mining text analysis and all of that let’s look at a couple more applications sentimental analysis is one of the major applications of natural language processing you see Twitter performs sentimental analysis Facebook Google all of these perform sentimental analysis sentimental analysis mainly used to analyze social media content
That can help us determine the public opinion on a certain topic then we have chat Bots now chat Bots use natural language processing to convert human language into desirable actions we also have machine translation NLP is used in machine translation by studying the morphological analysis of each word and
Translating it to another language advertisement matching is also done using NLP in order to recommend ads based on your history right these are few of the applications of NLP now let me tell you the basic terminologies under natural language processing so tokenization is the most basic step in natural language processing tokenization
Means breaking down the data into smaller chunks or tokens so that they can be easily analyzed so the first step is you’ll break a complex sentence into words then you’ll understand the importance of each of the word with respect to that sentence in order to produce a structural description on an
Input sentence so for example take this sentence how would I perform tokenizations on this sentence let’s say that tokens are simple is a sentence and I want to perform tokenization on the sentence this is what I’m going to do I’m going to split the sentence into different words I’m going to understand
Each word with respect to that sentence right this is done to simplify operations in natural language processing right it’s always simpler to analyze a single token instead of analyzing an entire sentence then we have something known as stemming now look at this example right here we have words such as detection detecting
Detected and detections we all know that the root word for all of these words is detect so stemming algorithm basically does that it works by cutting off the end or the beginning of the word and taking into account a list of common prefixes and suffixes that can be inflected word stemming basically helps
Us in analyzing a lot of words we know that detection is detected and detection basically mean the same thing so all we’re doing is we’re going to ease our analysis by removing prefixes and suffixes which not make sense right we just need to understand the morphological analysis of the word right
So that’s why we’re randomly cutting the prefixes and suffixes in such a way that we only get the important part of the word this is called stemming now this cutting of words can be successful in some occasions but not always that is why we say that stemming approach has a
Few limitations in order to get over these limitations we have a process known as limitization right limitization on the other hand takes into consideration the morphological analysis of the words it does not randomly cut the word in the beginning and the ending it understands what the word means and
Only then it cuts the word for example let’s consider the word recap if we perform stemming on the word recap we’ll get cap right the output will be cap but cap and recap do not have the same meaning do they they have absolutely different meanings that’s why stemming
Is sometimes not considered to be the right thing to do but when it comes limitization it’s going to understand the meaning of recap only then will it perform any sort of change in the word or it will cut down the word so basically it groups together different
Inflected forms of a word called Lemma lematization is similar to stemming because it Maps several words into one common root but the output of a limitization process is always a proper word an example of limitization is to map gone going and went into go gone going went all of them mean go so
Basically by limitization you can just output the words as go that is what limitization is next we have something known as stop words right stop words are basically a set of commonly used words in any language right not just English any language the reason why stop words are critical to many applications is
That if we remove the words that are very commonly used in a given language we can finally focus on the import important words for example in the context of let’s say you open up Google and you look for strawberry milkshake recipe instead of typing strawberry milkshake recipe let’s say you type how
To make strawberry milkshake now here what Google will do is it will find results for how to and make instead if you just type strawberry milkshake recipe you’ll get the most desired output that’s why it’s always considered a good practice in natural language processing to get rid of stop words
Right stop words will just increase our computation and it’ll just add additional work to us they are not very helpful when we are analyzing important documents right we need to focus on the important keywords in the documents instead of all of these commonly used words example of stop words include the
How when why not yes no all of these are stop words right so in order to better analyze our data we need to get rid of stop words now the last terminology I’m I’m going to discuss is document term Matrix it is important to create something known as The documenter Matrix
In natural language processing a DTM or a document term Matrix is basically a matrix that shows the frequency of words in a particular document let’s say that we’re trying to understand if the sentence this is fun is available in one of my documents so if it is there in my
Document one I’m going to put a 1 corresponding to each of the words that is available in my document for example in document 2 I have this is but I do not have the word fun similarly in document 4 I have the word this but I do not have the word is and
Fun so basically your document term Matrix is like the frequency Matrix of a document so during text analysis you always begin by building a document term Matrix right here you try to understand which words frequently occur and which words are important and not important in the document so Guys these were a couple
Of terminologies in natural language processing Thank you so let us understand when we have machine learning why do we need deep learning database will look at various limitations of machine learning now the first limitation is high dimensionality of the data now the data that is now generated is huge in size so we have a
Very large number of inputs and outputs so due to that machine learning algorithms fail so they cannot deal with high damage stability of data or you can say data with large number of inputs and outputs now there’s another problem as well in which it is unable to solve The
Crucial AI problems which can be natural language processing image recognition and things like that now one of the biggest challenges with machine learning models is feature extraction now let me tell you what are features since statistics we consider features as variables but when we talk about artificial intelligence these variables
Are nothing but the features now what happens because of that the complex problems such as object recognition or handwriting recognition becomes a huge challenge for machine learning algorithms to solve now let me give you an example of this feature extraction suppose if you want to predict that
Whether there will be a match today or not so it depends on our various features it depends on the whether the weather is sunny whether it is windy all those things so we have provided all those features in our data set but we have forgot one particular feature that
Is humidity and now our machine learning models are not that efficient that they will automatically generate that particular feature so this is one huge problem or you can say limitation with machine learning now obviously we have limitation and it won’t be fair that if I don’t give you the solution to this
Particular problem so we’ll move forward and understand how deep learning solves these kind of problems now as you can see that the first line on your slide which says that deep learning models are capable to focus on the right features by themselves requiring little guidance from the programmer so with the help of
Little guidance what these deep learning models can do they can generate their features on which the outcome will depend on at the same time it also solves the dimensionality problem as well if you have very large number of inputs and outputs you can make use of a deep learning algorithm now what exactly
Is deep learning again since we know that it has been evolved by Machine learning and machine learning is nothing but a subset of artificial intelligence and the idea behind artificial intelligence is to imitate the human behavior the same idea is for the Deep learning as well is to build learning
Algorithms that can mimic brain now let us move forward and understand deep learning what exactly it is now the Deep learning is implemented with the help of neural networks and the idea or the motivation behind neural networks are nothing but neurons what are neurons these are nothing but your brain cells
Now here’s a diagram of neuron so we have dendrites here which are used to provide input to our neuron as you can see here we have multiple dendrites here so these many inputs will be provided to a neuron now this is called cell body and inside the cell body we have a
Nucleus which performs some function after that that output will travel through Exon and it will go towards the Exon Terminals and then this neuron will fire this output towards the next neuron now the studies tell us that the next neuron now or you can say the two neurons are never connected to each
Other there’s a gap between them so that is called a synapse so this is how basically a neuron works like and on the right hand side of your slide you can see an artificial neuron now let me explain you that so over here similar to neurons we have multiple inputs now
These inputs will be provided to a processing element like our cell body and over here the processing element what will happen summation of your inputs and weights now when it moves on then what will happen this input will be multiplied with our weights so in the beginning what happens these weights are
Randomly assigned so what will happen if I take the example of X1 so X1 multiplied by W1 will go towards the processing element similarly X2 and W2 will go towards the processing element and similarly the other inputs as well and then summation will happen which
Will generate a function of s that is f of s after that comes the concept of activation function now what is activation function it is nothing but in order to provide a threshold so if your output is above the threshold then only this neuron will fire otherwise it won’t
Fire so you can use a step function as an activation function or you can even use a sigmoid function as your activation function so this is how an artificial neuron looks like so a network will be multiple neurons which are connected to each other will form an artificial neural network and this
Activation function can be a sigboid function or a step function that totally depends on your requirement now once it exceeds the threshold it will fire after that what will happen it will check the output now if this output is not equal to the desired output so these are the
Actual outputs and we know the real outputs so we’ll compare both of that and we’ll find the difference between the actual output and the desired output on the basis of that difference we are again going to update our weights and this process will keep on repeating
Until we get the desired output as our actual output now this process of updating weight is nothing but your back propagation method so this is neural networks in a nutshell so we’ll move forward and understand what are deep networks so basically deep learning is implemented by the help of
Deep networks and deep networks are nothing but neural networks with multiple hidden layers now what are hidden layers let me explain you that so you have inputs that comes here so this will be your input layer after that some process happens and it’ll go to the next
Node or you can say to the hidden layer nodes so this is nothing but your hidden layer one so every node is interconnected if you can notice after that you have one more hidden layer where some function will happen and as you can see that again these nodes are
Interconnected to each other after this hidden layer 2 comes the output layer and this output layer again we are going to check the output whether it is equal to the desired output or not if it is not we are again going to update the weights so this is how a deep Network
Looks like now there can be multiple hidden layers there can be hundreds of hidden layers as well but when we talk about machine learning that was not the case we were not able to process multiple hidden layers when we talk about machine learning so because of deep learning we have multiple hidden
Layers at once now let us understand this with an example so we’ll take an image which has four pixels so if you can notice we have four pixels here among which the top two pixels are bright that is their black and color whereas bottom two pixels are white now what happens we’ll divide
These pixels and we’ll send these pixels to each and every node so for that we need four nodes so this particular pixel will go to this node it will go to this node this pixel will go to this node and finally this pixel will go to this particular node that I’m highlighting
With my cursor now what happens we provide them random weights so these white lines actually represent the positive weights and these black lines represents the negative waves now this particular brightness when we display High brightness we’ll consider it as negative now what happens when you see
The next output or the next hidden layer it will be provided with the input with this particular layer so this will provide an input with positive weight to this particular node and the second input will come from this particular node since both of them are positive so
We’ll get this kind of a node similarly this node as well now when I talk about these two nodes the first node over here so this is getting input from this node as well as from this node now over here we have a negative weight so because of
That the value will be negative and we have represented that with black color similarly over here as well we’re getting one input from here which has a negative weight and the another input from here which has again has a negative weight so accordingly we get again a
Negative value here so these two becomes black in color now if we notice what will happen next we’ll provide one input here which will be negative and a positive weight which will be again negative and this will be also negative at a positive weight so that will again
Come out to be negative so that is why we have got this kind of a structure if you notice this this is nothing but the inverse of this particular image when I talk about this node over here we are getting the negative value with a positive weight which is negative and a
Negative value with a negative weight which is positive so we are getting something which is positive here now obviously I want this particular image to get inverse I want these black strips to come up so what I’ll do I’ll actually calculate the inverse by providing a negative weight like this so over here
I’ve provided a negative weight it will come up so when I provide a positive weight so it’ll stay wherever it is after that it will detect and the output you can see will be a horizontal image not a solid not a vertical not a diagonal but a horizontal and after that
We are going to calculate the difference between the actual output and the desired output and we are going to update the weights accordingly now this is just an example guys so guys this is one example of deep learning where what happens we have images here we provide
These raw data to the first layer to the input layer then what happens these input layers will determine the patterns of local contrast or it’ll fixate those patterns of local contrast which means that it will differentiate on the basis of colors and luminosity and all those things so it’ll differentiate those
Things and after that in the following layer what will happen it will determine the phase features it will fixate those phase features so it’ll form nose eyes ears all those things then what will happen it will accumulate those correct features for the correct phase or you
Can say that and fix it those features on the correct base template so it’ll actually determine the faces here as you can see it over here and then it will be sent to the output layer now basically you can add more hidden layers to solve more complex problems for example if I
Want to find out a particular kind of face for example a phase which has large eyes or which has light complexion so I can do that by adding more hidden layers and I can increase the complexity also at the same time if I want to find which
Image contains a dock so for that also I can have one more hidden layer so as and when hidden layer increases we are able to solve more and more complex problem so this is just a general overview of how a deep Network looks like so we have
First patterns of local contrast in the first layer then what happens we fixate these patterns of flow will contrast in order to form the phase features such as eyes nose ears Etc and then we accumulate these features for the correct phase and then we determine the
Image so this is how a deep Learning Network or you can say deep Network looks like and I’ll give you some applications of deep learning so here are a few applications of deep learning it can be used in self-driving cars so you must have heard about self-driving
Cars so what happens it’ll capture the images around it it will process that huge amount of data and then it’ll decide what actions it takes to take left right should It Stop So accordingly it’ll decide what action should it take and that will reduce the amount of accidents that happens every year then
When we talk about voice control assistance I’m pretty sure you must have heard about Siri all the iPhone users know about Siri right so you can tell Siri whatever you want to do it’ll search it for you and display for you then when you talk about automatic image
Caption generation so what happens in this whatever image that you upload the algorithm is in such a way that will generate the caption accordingly so for example if you have say blue colored eyes so it will display updo color I caption at the bottom of the image now
When I talk about automatic machine translation so we can convert English language into Spanish similarly Spanish to French so basically automatic machine translation you can convert one language to another language with the help of deep learning and these are just few examples guys there are many many other
Examples of deep learning it can be used in game playing it can be used in many other things and let me tell you one very fascinating thing that I’ve told you the beginning as well with the help of deep learning MIT is trying to predict future so yeah I know it is
Growing exponentially right now guys foreign Technology which is related to computer vision and image processing here what we do in case of object detection is for a given image I am going to check what are the objects that are present in the image and not only what I am also going to say where exactly is
That object is present so that is what happens in place of object detection an excellent example can be is Facebook photo tagging so you might have observed the Facebook photo tagging feature so when you upload a photo to your Facebook so the Facebook will automatically recognize the people
Who are present in that particular uploaded image and along with that it will also do one important thing along with that it will also show you as where exactly that person is present so if they like it is going to draw bounding box on top of that person’s
Face to show it as hey this is your friend a and he’s present over here do you want to tag him so that functionality that you would see in the Facebook uh in that functionality that you see so that is a typical example of object detection now I’m not
Sure what is the latest update because it’s been more than two years me using social media but yeah so two years back that was it and I’m sure today it will be the same feature or it will be an upgraded feature but the main concept is this when you upload something for your
Facebook or any social media I’m giving a Facebook as a generalized concept because that’s where I’ve seen so it is going to tell us who are all the people that are present in the image and also it is going to tell us as where that person is present and that is what in
Object detection algorithm is going to do it’s going to identify the objects in the objects in a given image and it also tells us as where exactly that object is present so what and where both of them will be identified in case of object detection so here you can see the GIF animation
That you’ll see over here on your on your left hand of the screen so here it says what are the objects that are present in the image so the objects are capped bowel Pepsi can and so on now along with saying what are the object it
Is also drawing a bounding box so you can see a green box that has been drawn on top of each and every object to tell us as where that particular object is present and here on the another image that we have on the right hand of the slide so here uh it
Also it’s also doing a similar activity it is saying there’s a retriever dog there is an American stand for terrier dog and there is just a common dog okay and there is a a built terrier dog so there are so it has identified the dogs that are present in the image and along
With identifying what is the breed of the dog it is also drawing a box see it says this is where my dog is this is where my dog is this is where my dog is so this is how any typical object detection application would work so it detects what and also it also
Tells us as where exactly that particular object is present now the next question is where do we see these applications so one obvious face recognition so this we have been familiar with Facebook so where we upload an image and we can identify who are all the people that are present in the image
And even Google photos as a good face recognition feature I mean so we’ve been understanding about this and even the Google photos has a good uh face recognition or yeah first recognition over there so when we upload all the pictures so it’s going to group all the photos together and it will ask
Us to name that particular one person and it will automatically tag those names to all the similar people wherever that person is present so that’s also an application of face recognition and we can also use this object detection concept for the people counting to understand as how many people are
Present over there in a given scenario or another example can be is uh whether that person is wearing a mask or not so wearing a mask or not so I can do it and I can also I can also use this visual uh ma I can also use this mask
Detection object detection for checking whether people are following the Kuwait 19 protocols that is social distancing so all these are the applications of using object detection and even we can use it for industrial purpose it could be for industrial quality checking to check whether the objects that has
Been done is a object that is being manufactured is it as per these standards are being expected by the industry or self-driving cars so in in case of self-driving cars so the cars will identify what are the objects that are present in a given image okay or a given video feed and on
The basis of the object it’s going to drive so that’s another application of cell application of object detection and the another application that I could see is a security the face ID that we have in our Apple iPhones so that uses the object detection to identify who is present in
That image and another thing is like identifying the objects on the road I think you might have seen uh uh you might have seen the latest Mahindra XUV 700 so they they have an functionality to detect the uh like they have the functionality to detect the objects that
Are in front of in front of the car and if the of the object is very close to the car it is going to automatically apply the brake now here the object detection obviously uh like to detect the objects in front of the car so they are making use of object detection over
There so that’s another application okay so these are some of the applications of using object detection in real life now coming to this object detection let’s look at the typical workflow and how does it work when we want to build an object detection algorithm using tensorflow when we are working with the
Applications like object detection first we need to prepare the training data I’ll have to build a training data to such that to identify whether my given image has the required number of classes so I’ll have to build my classifier to identify as what is that individual object is present or what is my
Individual object contains now after I have trained this so this is my training data I have the images of cars and I have the images of bike and I have trained my machine learning model to identify whether my given image contains this car and bike now once it’s rained I can obviously go
Ahead and send in my test data and with that test data I can detect whether my given input image contains bike or car okay so this is the first step in preparing the object detection okay first I’ll build a classifier which will help me in identifying what is present in the image
Now along with this what while training I’ll also tell my machine learning or in this case deep Learning System to tell us where exactly that object is present so my machine learning model or this in this scenario it’s a deep learning model my deep learning model will do two
Things in parallel one it will identify what and the another thing it will also do is where it will learn where exactly that object is present in that given image so it does both the things in parallel so what happens is like I’ll have some set of layers over here and from this
Layer I another set of another another series of layers for detection to identify what is present over there and another set of layers for the text for detecting where exactly that object is present okay so that’s what it will be the workflow of object detection now once I have trained my object
Detection model I can send in a new image now when I send a new image my deep learning model will be able to identify what is present in the image and also it also give tells me as where exactly that object is present in a given image and that’s how an object
Detection machine learning algorithm would work now in order to build this deep learning models which is capable of Performing object detection we have various Frameworks now one of the common framework that we would use in Suite tensorflow so this is one of the very common uh framework that
We are currently using which are in the latest scenario because of its uh ease of deployment using the tensorflow extend so because of this functionality so now Industries are preferring to use this tensorflow framework now apart from this we have Pi torch and makes that theano so these are the
Various Frameworks that are also available to us to build deep learning neural network now let’s get an understanding about this tensorflow and what does this tensorflow is made of now as I mentioned already tensorflow is a deep learning framework which helps us in building the Deep learning neural
Network now just like my numpy library has the array object which is one of the basic uh basic objects from this numpy library in case of tensorflow library or you can take any deep learning Frameworks we have an object that’s called as tensors now these tensors are the standard way
Of representing the data in case of deep learning the reason that we used to prefer representing the data with the tensors because one it will help us to process the data with GPU GPU means graphical Processing Unit which we use it for gaming purpose okay you know already what is computer gaming
So one like to in order to get the speed and in order to get the good graphics we know that we actually use the best best and the best uh graphics card so that we can play the game in a very high setting Now using this deep learning Frameworks
And by creating the data inside that tensors I’ll be having the ability I mean I’ll be having the ability to process those data that I have using these GPU cores so just like we have a course in CPU we’ll also have the processing course in GPU and we’ll be
Utilize those GPU cores and we have seen that we could get around 10 to 20 x faster than we could than what we could achieve in our CPU so that’s the advantage of working with deep learning Frameworks and creating the data as tensors and if I have a Cuda supported
GPU then I can also make use of this GPU course to get 10 to 20 times of faster data processing compared to CPU so that’s the reason we we want to gen we want to represent the data in terms of tensors now what is this tensor sensors are just
The multi-dimensional array object it’s just an extension of two dimensional tables uh two data with higher dimension so that’s what these tensors are now in this tensorflow the computation is approached as a data flow diagram so whenever I want to do anything over there in tensorflow so normally what we
Do if you remember in case of any machine learning model I’m going to represent how my y hat is computed and then from my y hat I’m going to calculate the cost so I’ll have to manually specify how the how the cost should be done and I have to mention as
How the gradient should be calculated with reference to every parameter that I am that I am currently having in my model but in case of tensorflow I don’t have to worry about it I can let my tensorflow to do the magic I can let my tensorflow to track what
Are the operations that I’m doing on the data and once I have completed the operation I can just say tell to my system as okay hey I have finished doing the operation why don’t you update the parameters so in order to do that we actually make
Use of data flow graph or sometimes we also called as tensorflow graph so it’s going to track how the data is getting manipulated now once I have done this data manipulation I can make use of this tensorflow graph to update the parameters of my model during the operation and that’s the advantage of
Using deep learning framework now because of this feature I don’t have to worry about writing the complex equation I can just start building the models which are as as much as complex that I want and I can let my tensorflow to do the magic and find the gradients and
Update the parameters even though with just a single command so that’s the advantage of working with this tensorflow and we have the similar functionality in pi torch Library so it’s not just advantage in tensorflow I’m just giving it as an example as tensorflow because we are discussing a tensorflow the similar functionality is
Also available in pi torch as well okay and yes we represent the data in tensors and once we represent the data intense us so here it is just a flow over here so here it the flow says okay for the given date I am going to add something and I’m going to perform
Matrix multiplication so matrimal is nothing but the matrix multiplication so I’m doing some matrix multiplication and I’m getting the result and on the basis of result if I want to update something I can just go ahead and perform some update using the tensorflow data graph that I have and then find the parameters
Over there and I can do my required activity as per my preference so what we do in case of building the model is we start with the input data we are going to convert it into the tensors and once we convert the data into the tensors using tensorflow Library we’ll
Start training the model with tensorflow model okay so here the tensorflow model is nothing but the Deep learning model which I’ve created with tensorflow Library now once that is done I can go ahead and look like start training the model in depth so here I’ll create it as tensors and then I’m going
To train my tensorflow model which is a deep learning model and once the model has been trained I can send in a new data which is called as test data to test how my model is performing and once the testing is done once I’m satisfied with the output obviously I
Can use those test results to tell us as okay so this is an example of object detection so for a given image this is where the objects are and this is these These are the objects so person dog and horse so there are three objects that are present over here
And the bounding box tells us as where exactly those objects are present so this is a typical flow when it comes to performing object detection now here as a Hands-On demo we are going to look into our demo of YOLO model okay so YOLO is one of the uh most commonly
Used object detection model when it comes to object performing the object detection using deep learning models so it’s an object detection models we’ll use this already trained YOLO model and we’ll send in our data and we’ll see how the output would look like when I perform object object detection with this yellow
Okay so let’s go back to our Google collab okay now let’s have a quick look at the implementation of tensorflow so this is a notebook that I’ve just shared in the chat window you can refer that notebook and this notebook has the implementation of object detection with yellow
Now here to get started we are going to get the repository from the GitHub so this is going to clone my repository which is present over there that is The Dark Knight repository and then I’m going to go inside the directory that is cid.net and this is going to display the
Various files that we have in that directory now once that is done we are going to execute this command of make so this will compile my library and this is going to make sure that it is ready for us to execute so this would take a while
Depending on the computer speed that we are running in now in this scenario I’m using the free system that is provided by Google collab and this would take a while uh for for it to become ready and once that is done I’m going to get the trained weights or already trained
Weights from the GitHub repository so this is actually available under this link so I’m going to download those weights so here the weights are nothing but the already trained data so I’ll use the already trained data and I would get that data and once I get the data I can just go
Ahead and test my object detector now here for the testing purpose I’m going to make use of the image that is called as persons image which is present inside my data directory okay I’ll show you what is that image that I’m talking about so data so there are various images now this
Time I’m using this image of person so this is the image of person so on this image I’m going to identify what are the objects that are present in this given image so let’s see what will be the output if I just execute this line so this is
Going to tell me this would take a while to perform the prediction okay and then it will give us the detection okay so we have the output and if You observe over here currently I’m running on CPU so it actually took 21 915 milliseconds so I’ll just note this
Number in a separate Notepad so this is the military this is the 21 050 Milli seconds it took on my CPU all right and if I come down so this is the uh predictions that has been given from my object detection model of YOLO so it says where exactly that person is
President it also displays about the uh class name here the class name is says as person and along with that it also displays The Confidence Code so how much confidence uh to uh how much continent it is to this saying it as okay it belongs to the person
Okay and it says 98 confidence that belongs to the horse and this is 99 confident that it belongs to the dog classes now we are going to enable the GPU and once we enable the GPU we’ll have to execute some commands again and prepare our model to make sure that it runs on
GPU and once that is done this time we are going to use a new image the new image is called as giraffe.jpg so this is the image that I’ll be testing out so this is the image of giraffe and I’ll be using this image to detect the
Objects so I’ll just execute this to run the prediction and I’m going to show how the objects would look like in this image okay it’s still configuring my system to make sure that it runs on the GPU now the system setup is complete now let’s look at the logs now if I come
Down and look at the logs it just took less than a second to complete the prediction so it just took around 165 milliseconds see you can look into the comparison now if I just compare it okay to 1915 divided by 165 so it’s like 132 times
Faster than what I have got from my CPU so that’s the computation speed that we would get when we work with uh gpus okay now if I see over here so it says these are the object that has been identified from a given image so there’s a giraffe
With 100 confidence and there is a zebra with 99 confidence so clearly we would have we would get the high speed computation when we are working with gpus now below we have a code to test out about the various threshold values so for that we are
Choosing an example image of Hots and on this example image of odds so by default it has some threshold value I think the threshold value is somewhere around 0.8 or something now if I reduce the threshold value it is going to add some more images and if I increase the
Threshold values it is going to stop uh like displaying about the allows this in this example I have mentioned my threshold is 0.98 that means detect the objects for which you are confident with 98 percent so here in the above image I had an in I had an object which had 90 confidence
And that has been omitted when I specify this threshold of 98 percent and if I reduce the threshold value then in such scenarios it would randomly assign the values like this because the threshold is very less okay so that is how we can make use of tensorflow and
Start that’s how we can make use of tensorflow and detect the objects in an image and if I come down there is a code which talks about as we can perform the object detection in videos now instead of connecting to the videos we can also make use of opencv library and detect
The starter webcam session and then we can run this run this object detection on the video file as well or we can run that object detection on the in on the webcam Sim as well to detect what are the objects that are present in the image now in order to do
That obviously we need a system which has a very good memory and the GPU and you are free to check out at your end when you’re exploring about videos okay so the overall the identification or the overall understanding that you need to have is when I talk about object detection
So object detection means for a given input data I’ll be doing two things one I’ll be detecting where is my object is present foreign where is my object and what is my object I’ll detect both the things when I’m performing object detection so what and where which is La
Which is like what we have seen in this video so in our current scenario so we have made use of YOLO object detector to detect the objects from our end and when you start your Learning Journey with tensorflow obviously you learn how you can create your own object detection
Model to identify the objects in a given image so you learn that as well as you progress in your Learning Journey when you start exploring tensorflow and when you try to do some tasks on your own okay so that’s how you can plan out in your Learning Journey So let’s understand how exactly a computer reads an image so this is an image of New York skyline I personally love this picture so when a human will see this image he’ll first notice there are a lot of buildings in different colors and stuff like that but how a
Computer will read this image so basically there will be three channels one will be red another will be green and finally we have blue Channel which is popularly known as RGB so each of these channels will they have their own respective pixel values as you can see
It over here so when I say that image size is B cross a cross 3 it means that there are B rows a columns and three channels all right so so if somebody tells you that the size of an image is 28 cross 28 cross 3 pixels it means that
It has 28 rows 28 columns and three channels so this is how a computer sees an image and this is for colored images for black and white images we have only two channels so let’s move forward and we’ll see why can’t we use fully connected networks for image classification so consider an image
Which has 28 cross 28 cross 3 pixels so when I feed in this image to a fully connected Network like this then the total number of ways required in the first layer will be 2352. you can just go ahead and multiply it yourself all right but in real life the images are
Not that small all right so whatever images that we have they are definitely above 200 cross 200 cross 3 pixels so if I take an image which has 200 cross 200 cross 3 pixels then I feed it to a fully connected Network so at that time the
Number of bits required the first hidden layer itself will be 120 000 guys so we need to deal with such huge amount of parameters and obviously we require more number of neurons so that can eventually lead to overfitting so that’s why we cannot use fully connected Network for
Image classification now let’s see why we need convolutional neural networks so basically in convolutional neural network a neuron in the layer will only be connected to a small region of the layer before it so if you consider this particular neuron which I’m highlighting right now is only connected to three
Other neurons unlike the fully connected Network where this particular neuron will be connected to all these five neurons because of this we need to handle less amount of weights and in turn we need less number of neurons as well so let us understand what exactly is convolutional neural network so can
Evolutional neural networks are special type of feed forward artificial neural networks which is inspired from visual cortex so visual cortex is nothing but a small region in our brain brain which is present somewhere here where you can see the bulb and basically what happened there was an experiment conducted and
People got to know that visual cortex has small regions of cells that are sensitive to specific regions of visual field so what I mean by that is for example some neurons in the visual cortex fires when exposed to Vertical edges some will fire when exposed to horizontally the some will fire an
Exposed to diagonal edges and that is nothing but the motivation behind convolutional neural networks so now let us understand how exactly a convolutional neural network works so generally a convolutional neural network has three layers convolution layer layer pooling layer and fully connected layer we’ll understand each of these layers
One by one we’ll take an example of a classifier but that can classify an image of an X as well as an O So with this example we’ll be understanding all these four layers so let’s begin guys now there are certain trickier cases so what I mean by that is X can be
Represented in these fourth forms as well right so these are nothing but the deformed images effects similarly for o as well so these are deformed images so even I want to classify these images either X or o all right because even this is X this is X this is X this is X
But all these are deformed images but they are in turn X right so I want my classifier to classify them as X so basically that’s what I want so if you can notice here this is a proper image of an X and which is actually equal to
This particular X which is a deformed image same goes for this o as well so now what we are going to do is we know that a computer understands an image using numbers at each pixels so what we’ll do whatever the white pixels that we have we are going to assign a value
-1 to it and whatever black pixels we have we are going to assign a value 1 to it when we use normal techniques to compare these two images one is a proper image of X and another is a deformed image of X we got to know that a
Computer is not able to classify the deformed image of X correctly why because it is comparing it with the proper image of X right so when you go ahead and add the pixel values of both of these images you get something like this so base basically our computer is
Not able to recognize whether it is NX or not now what we do with the help of CNN we take small patches of our image so these patches or these pieces are known as nothing but features or filters so what we do by finding rough feature matches in roughly the same positions in
Two images CNN gets a lot better at seeing the similarity between the whole image matching schemes what I mean by that is we have these filters right we have these filters that you can see so consider this first filter this is exactly equal to the feature of the part
Of the image in the deformed image as well so this is a proper image and this is our deformed image all right so this particular feature or this particular part of the image is actually equal to this particular part of the image same goes for this particular feature of
Filter as well and similarly we have this filter as well which is actually equal to this particular part of the deformed image all right so let’s move forward and we’ll see what all features that we’ll be taking in our example so we’ll be considering these three features or filters this is a diagonal
Filter this is again a diagonal filter and this is nothing but a smaller so we’ll take these three filter and we’ll move forward so what we are going to do is we are going to compare these features the small pieces of the bigger image we are going to put it on the
Input image and if it matches then the image will be classified correctly now we’ll begin guys the first layer is convolution layer so these are the beginning two steps of this particular layer first we need to line up the feature in the image and then multiply image by the corresponding feature picks
It now let me explain you with an example so this is our first diagonal feature that we’ll take we are going to put this particular feature on our image of X all right and we’re going to multiply the corresponding pixel value so one will be multiplied with one we’ll
Get one and we’ll put it in another Matrix similarly we are going to move forward and we are going to multiply minus 1 with minus 1 we’re going to multiply minus 1 with minus 1 as you can see similarly we multiply this result minus 1 into minus 1 then again minus 1
Into minus 1 so we are going to complete this whole process when we are going to finish up this Matrix all right and once we are done finishing up the multiplication of all the corresponding pixels in the feature as well as in the image we need to follow two more steps
We need to add them up and divide by the total number of the pixels in the feature so what I mean by that is after the multiplication of the corresponding pixel values what we do we add all these values we divided by the total number of
Pixels and we get some value right and then now our next step is to create a map and put the value of the filter at that particular place we saw that after multiplying the pixel value of a feature with the corresponding pixel value of with that of our image we get the output
Which is one so we place one here similarly we are going to move this filter throughout the image next up we are going to move this filter here after that we are going to move it here here here everywhere on the image we are going to move it and we are going to
Follow the same process all right so yeah this is one more example where I’ve moved my filter in between and after doing that I’ve got the output something like this 1 1 minus 1 and all so over here if you notice I’ve got couple of
Times minus 1 as well due to which my output that comes is 0.55 right so I’m gonna Place 0.55 here similarly after moving the pixel after moving the filter throughout the image I got this particular color Matrix all right and this is for one particular feature after performing the same process for the
Other two filters as well I’ve got these two values so we have these three values after passing through the convolution layer let me give you a quick recap of what happens in convolution layer so basically we have taken three features all right and one by one we’ll take one
Feature move it through the entire image and when we are moving it at that time we are multiplying the pixel value of the image with that of the corresponding pixel value of the filter adding them up dividing by the total number of pixels to get the output so when we do that for
All the filters we get we got these three outputs all right so let’s move forward and we’ll see what happens in relu layer so this is Railway layer guys and uh people who have gone through the previous tutorial actually know what it is so let me just give you a quick
Introduction of relu layer so relu is nothing but a activation function all right so what I mean by that is it will only activate a node if the input is above a certain quantity while the input is below zero the output is also zero all right and when the input rises above
The certain threshold it has a linear a relationship with a dependent variable now I’ll explain you with an example if we have a graph of relu function here so my function says that when f of x is equal to 0 if x is less than 0 and it is
Equal to X when X is greater than 0 all right so whatever values that I have which are below zero will actually in turn become 0 and whatever values that are above 0 our function value will also be equal to that particular value so f
Of x will be equal to X if it is greater than or equal to 0 and it will be 0 if it is less than 0. so if I have x value as minus 3 so definitely it is less than 0 so f of x becomes 0. similarly if I
Have minus 5x value then that again it is less than 0 so my f of x value becomes 0 but when I consider 3 as my x value then my f of x becomes equal to X which is nothing but 3. so over here
I’ll have 3. again if I take my x value as 5 then obviously it is greater than or equal to 0 then my f of x becomes equal to X so my f of x value becomes 5. so this is how a relu function works so
Why are we using relu function here is we want to remove all the negative values from our output that we got through the convolution layer so we’ll only take the first output that we got by moving one feature throughout the image so this is the output that we have
Got verb only one filter all right so over here I’m going to remove all negative values so over here you can see that it it was minus 0.11 before and I’ve converted that to zero similarly I’m going to repeat the whole process for the entire Matrix and once I’m done
With that I get this particular value now remember this is only for the output that we got through one feature all right so when we were doing convolution at that time we were using three features right so this is output only for one filter after doing it for the
Output of the other two filters as well we have got these two values more so totally we have these three values after passing through railroad activation function next up we’ll see what exactly is pulling left so in pooling layer what we do we take a window size of 2 and we
Move it across the entire Matrix that we have got after passing through relu layer and we take only the maximum value from that so that we can shrink the image so what we are actually doing doing is we are reducing the size of our image so let me explain you with an
Example so this is basically one output that we have got after passing through relu layer and over here we have taken a window size of 2 cross 2 so when we keep this window at this particular position we see that one is the highest value so
We are going to keep one here and we are going to repeat the same process for this particular window as well so over here the maximum value is 0.33 so 0.33 will come so if you notice here earlier we had 7 cross 7 Matrix and now we have reduced
That to 4 cross 4 Matrix so after doing that for the entire image we have got this as our output this output we have got after moving our window throughout the image that we have got after passing through relu layer right and when we repeat this process for all the three
Outputs that we have got after the relu layer then we get this particular output after polling layer right so basically we have shrinked our image to a four Cross Four Matrix now comes the tricky part so what we are going to do now is stack up all these layers so we have
Discussed convolution layer relu layer and pooling layer so I’ll just give you a brief recap of what all things we have discussed in convolution layer what we did we took three features and then after that one by one we moved each filter throughout the image and when we
Were moving it we are continuously multiplying the image pixel value with that of the corresponding filter pixel value and then we were dividing it by the total number of pixels all right with that we got three output after passing through the convolution layer then those three output we pass through
A relu layer where we have removed the negative value all right and after removing negative value again we have got the three outputs then those three outputs we pass through polling layers so basically we’re trying to shrink our image and what we did we took a window
Size of two cross two moved it through all the three outputs that we have got through relu layer and after doing that we were only taking the maximum value pixel value in that particular window and then we were putting it in a different Matrix so that we get a shrink
Damage and after passing it through pooling near we’ve got a 4 cross 4 Matrix and since we took three features at the beginning so therefore we have got the three outputs after passing through pooling lab all right next up we are going to stack up all the layers all
Right so let’s do that so after passing through convolution Railway and pooling we have got this four Cross Four Matrix this was our input image now when we add one more layer of convolution relu and pooling we have shrinked our image from 4 cross 4 to 2 cross 2 as you can notice
Here now we are going to use fully connected layer now what happens in fully connected layer the actual classification happens here guys okay so what we have doing here is we are going to take the shrinked images and put it into a single list so basically this is
What we have got after passing through two layers of convolution Railway and pooling and this is what we have got so basically we are converting into a single list or a vector how we do that we take the first value one then we take 0.55 then we take 0.55 then we take one
Again then we take one then we take 0.55.5 point five five point five five then we again take 0.5511 and 0.55 so this is nothing but a vector or you can say a list if you notice here that there are certain values in my list which has high for x
And similarly if I repeat the entire process that we have discussed for o there’ll be certain different values which will be high so for an X we have first fourth fifth tenth and 11th element a vector values are higher for o we have second third Ninth and 12th
Element Vector which are higher so basically we know now if if we have an input image which has a first fourth fifth tenth and 11th element Vector values High we know that we can classify it as X similarly if our input image has a list which has the second third ninth
And 12th element Vector values High then we can classify it as zero now let me explain you with an example so after training is done after the after doing the entire process for both X and O you know that our model is trained now okay
So we have given one a new input image and that input image passes through all the layers and once it has passed through all the layers we have got this 12 element Vector now it has 0.9.65 all these values right now how do we
Classify it whether it is an X or o so what we do we’ll compare this with a list of X and O right so we have got the list in the previous slide if you notice we have got two different lists for x and O we are going to compare this new
Input image list that we have got with that of X and O right so first let us compare that with X now as I’ve told you earlier as well for X there are certain values which will be high which is nothing but first fourth fifth tenth and
11th value right so I’m going to sum first 4th 5th 10th and 11th value and I’ve got five one plus one plus one plus one and plus one so five times one I’ve got five and now I’m going to sum the corresponding values of my input image
Vector as well so the first value is 0.9 then the fourth value is 0.87 fifth value is 0.96 tenth value is point eight nine and the eleventh value is 0.94 so after this doing the sum of these values I’ve got four point five six when I
Divide this by 5 I got 0.91 right now this is for X now when I do the same process for o so you know if you notice I have second third ninth and 12th element Vector values is high so when I sum these values I get 4 and when I do
The sum of the corresponding values in my input image I’ve got 2.07 when I divide that by 4 I got 4 4.51 so now we notice that 0.91 is a higher value compared to 0.51 so when we have compared our input image with the values
Of X we got a higher value then the value that we have got after comparing the input image with the values of four so the input image is classified as X all right so now let us move towards our use case so this is our use case guys so
Over here what we are going to do is we are going to train our model on different types of dogs and cat images and once the training is done we are going to provide it an input and it will classify whether the input is of a dog
Or a cat now let me tell you the steps involved in it so what we are going to do in the beginning is obviously first we need to download the data set after that we are going to write a function to encode the labels labels are nothing but
The dependent variable that we are trying to predict so in our training data and testing data obviously we know the labels right so on that basis only we can train our mode so we are going to encode those lab after that we’ll resize the image to 50 cross 50 pixel and we
Are going to read it as a grayscale image then we are going to split the data 24 000 images for training and 50 for testing once this is done we are going to reshape the data appropriately for tensorflow not tensorflow I think everyone knows about tensorflow tensorflow is nothing but a python
Library for implementing deep learning models then we are going to build the model calculate the loss it is nothing but categorical cross entropy then we are going to reduce the loss by using Adam Optimizer with a learning rate Setter aside to 0.001 then we are going to trade the Train the Deep neural
Network for 10 epox and finally we are going to make predictions all right so I’ll just quickly open my Python and I’ll show you the code how it looks like so this is the code that I’ve written in order to implement the use case in the beginning I need to import the libraries
That I require and once it is done what I’m doing I’m defining my training data and the testing data so train and test one contains both my training data as well as testing data respectively then I’ve taken my image size as 50 and learning rate I’ve defined here and I’ve
Given a name to my model you can give whatever name you want all right so first thing that we saw we need to encode the dependent variable that’s what we are doing here we are encoding our dependent variable so whenever the label is cat then it will be converted
To an area of one comma zero and when it is dog it will be converted to an array of zero comma one so by why we’re actually encoding the label because our code cannot understand the categorical variable so we need to encode it right next what I’m doing is I’m resizing my
Image to 50 cross 50 and I am converting it to a grayscale image right and once this is done I am going to split my data set into training and testing paths so yeah we are basically splitting the data set into two parts we for training and
Testing and here we are defining a model so you can just I can just go ahead and throw in a comment here building the model yeah so this is where we are building the model so basically what we have done here is we have resized our image to 50 cross 50 cross
One Matrix and that is the size of the input that we are using right this input that I’m talking about then what we have done a convolution layer we have defined with 32 filters and a stride of five with an activation function a relu and after that we have added a pooling layer
Max pool layer okay again what we have done we have repeated the same process but over here we are taking 64 filters and five stride passing it through a railroad activation function and after that we have a pooling lip match pool layer then we have repeated the same
Process for 128 filters after that we’ve repeated for 64 filters then for 32 filters then after that we are using a fully connected layer with 1024 neurons and finally we are using the Dropout layer with key probability of 0.8 to finish our models this is where our
Model is actually finished and then what we are doing is we are using the Adam Optimizer to uh optimize our model so basically whatever the laws that we have we are trying to reduce it and this is basically for your 10 support so we are creating some log files and then with
That log file tensorboard will create a pretty fancy grasp for us that helps us to visualize the entire model and then what we are doing is we are trying to fit the model and we have defined epochs as 10 that is the number of iterations
That will happen will be 10 and yeah so this is pretty much it modern name we have given then input is X underscore test to uh check the accuracy similarly uh the target will be y underscore test the labels associated with that test data will be a y underscore test and
Which we have encoded basically so this is how we are going to actually calculate the accuracy and we’ll try to reduce the loss as much as possible in 10 epochs so till now our model is complete we are done with it next what I’m doing is I’m feeding in some random
Input from the test data and I’m validating whether my model is predicting it correct or not all right so I’ve already trained the model because it takes a lot of time and yeah I cannot do it here so I’ve already trained the model and you can see that
The loss that came after the 10th Epoch is 0.2973 and the accuracy is somewhere around 88 percent which is pretty good guys and yeah and I’ve done the prediction on the test data as well so let me just show it to you that so this is the prediction that it has done on
Few of the images in the test data so yeah it is a cat predicted as cat cat predictors at cat cat cat and dogs as well there are certain dogs as well So this is the problem statement guys we need to figure out if the banknotes are real or fake and for that we’ll be using artificial neural networks and obviously we need some sort of data in order to train our Network so let us see how the
Data set looks like so over here I’ve taken a screenshot of the data set with few of the rows init data were extracted from images that were taken from genuine and forged banknote-like specimens after that wavelet transform tools were used to extract features from those images and these are few features that I’m
Highlighting with my cursor and the final column or the last column actually represents the label so basically label tells us to which class that pattern represents whether that pattern represents a fake note or it represents a real node let us discuss these features and labels one by one so the
First feature or the First Column is nothing but variance of a wavelet transformed image the second column is about skewness the third is courtesies of wavelet transformed image and finally fourth one is entropy of the image after that when I talk about label which is nothing but my last column over here if
The value is one that means the pattern represents a real load whereas when value is 0 that means it represents a fake node so guys let’s move forward and we’ll see what are the various steps involved in order to implement this use case so over here we’ll first begin by
Reading the data set that we have we’ll Define features and labels after that we are going to encode the dependent variable and what is the dependent variable it is nothing but your label then we are going to divide the data set into two parts one for training another
For testing after that we’ll use tensorflow data structures for holding features labels Etc and tensorflow is nothing but a python library that is used in order to implement deep learning models or you can say neural networks then we’ll write the code in order to implement the model and once this is
Done we will train our model on the training data we’ll calculate the error okay error is nothing but your difference between the model output and the actual output and we’ll try to reduce this error and once this error becomes minimum we’ll make prediction on the test data and will calculate the
Final accuracy so guys let me quickly open by pie chart and show you how the output looks like so this is my pycharm guys over here I’ve already written the code in order to execute the use case I’ll go ahead and run this and I’ll show you the output
So over here as you can see with every iteration the accuracy is increasing so let me just stop it right here so we’ll move forward and we’ll understand why we need neural networks so in order to understand why we need neural networks we are going to compare the approach before and after neural
Networks and we’ll see what were the various problems that were there before neural networks so earlier conventional computers use an algorithmic approach that is the computer follows a set of instructions in order to solve a problem and unless the specific steps that the computer needs to follow are known the
Computer cannot solve the problem so obviously we need a person who actually knows how to solve that problem and he or she can provide the instructions to the computer as to how to solve that particular problem right so we first should know the answer to that problem
Or we should know how to overcome that challenge or problem which is there in front of us then only we can provide instructions to the computer so this restricts the problem solving capability of conventional computers to problems that we already understand and know how to solve but what about those problems
Whose answer we have no clue of so that’s where our traditional approach was a failure so that’s why neural networks were introduced now let us see what was the snap are you after neural networks so neural networks basically process information in a similar way the human brain does and these networks they
Actually learn from examples you cannot program them to perform a specific task they will learn from their examples from their experience so you don’t need to provide all the instructions to perform a specific task and your network will learn on its own with its own experience all right so this is what basically
Neural network does so even if you don’t know how to solve a problem you can train your network in such a way that with experience it can actually learn how to solve the problem so that was a major reason why neural networks came into existence so we’ll move forward and we’ll understand what
Is the motivation behind neural networks so these neural networks are basically inspired by neurons which are nothing but your brain cells and the exact working of the human brain is still a mystery though so as I’ve told you earlier as well that neural networks work like human brain and so the name
And similar to a newborn human baby as he or she learns from his or her experience we want a network to do that as well but we wanted to do very quickly so here’s a diagram of a neuron basically a biological neuron receives input from other sources combines them
In some way perform a generally non-linear operation on the result and then output the final result so here if you notice these dendrites these dendrites will receive signals from the other neurons then what will happen it will transfer it to the cell body the cell body will perform some function it
Can be summation it can be multiplication so after performing that summation on the set of inputs via Exon it is transferred to the next neuron now let’s understand what exactly are artificial neural networks it is basically a Computing system that is designed to simulate the way the human brain analyzes and process the
Information artificial neural networks has self-learning capabilities that enable it to produce better results as more data becomes available so if you train your network on more data it will be more accurate so these neural networks they actually learn by example and you can configure your neural network for specific applications it can
Be pattern recognition or it can be data classification anything like that all right so because of neural networks we see a lot of new technology has evolved from translating web pages to other languages to having a virtual assistant to order groceries online to conversing with chat Bots all of these things are possible
Because of neural networks so in a nutshell if I need to tell you artificial neural network is nothing but a network of various artificial neurons all right so let me show you the importance of neural network with two scenarios before and after neural network so over here we have a machine and we
Have trained this machine on four types of dogs as you can see where I’m highlighting with my cursor and once the training is done we provide a random image to this particular machine which has a dot but this dog is not like the other dogs on which we have trained our
System on so without neural networks our machine cannot identify that dog in the picture as you can see it over here basically our machine will be confused it cannot figure out where the dog is now when I talk about neural networks even if we have not trained our machine
On this specific dock but still it can identify certain features of the dogs that we have trained on and it can match those features with the dog that is there in this particular image and it can identify that dot so this happens all because of neural networks so this
Is just an example to show you how important our neural networks now I know you all must be thinking how neural networks work so for that we’ll move forward and understand how it actually works so over here I’ll begin by first explaining a single artificial neuron
That is called as perceptron so this is an example of a perception over here we have multiple inputs X1 X2 dash dash dash till xn and we have corresponding weights as well W1 for X1 W2 for X2 similarly W and for xn then what happens we calculated the weighted sum of these
Inputs and after doing that we pass it through an activation function this activation function is nothing but it provides a threshold value so above that value my neuron will fire else it won’t fire so this is basically an artificial neuron so when I talk about a neural
Network it involves a lot of these artificial neurons with their own activation function and their processing element now we’ll move forward and we’ll actually understand various modes of this perceptron or single artificial neuron so there are two modes in a perceptron one is training another is using more in training mode the neuron
Can be trained to fire for particular input patterns which means that will actually train our neuron to fire on certain set of inputs and to not fire on the other set of inputs that’s what basically training mode is when I talk about using mode it means that when a
Taut input pattern is detected at the input its Associated output becomes the current output which means that once the training is done and we provide an input on which the neuron has been trained on so it will detect the input and will provide the associated output so that’s
What basically using mode is so first you need to train it then only you can use your perceptron or your network so these are the two modes guys next up we’ll understand what are the various activation functions available so these are the three activation functions although there are many more
But I’ve listed them three step function so over here the moment your input is greater than this particular value your neuron will fire else it won’t similarly for sigmoid and sine function as well so these are three activation functions there are many more that I’ve told you
Earlier as well so yeah these are the three majorly used activation functions next up what we are going to do we are going to understand how a neuron learns from its experience so I’ll give you a very good analogy in order to understand that and later on when we talk about a
Neural networks or you can say multiple neurons in a network I’ll explain you the math behind it I’ll explain you the math behind learning how it actually happens so right now I’ll explain you with an analogy and guys trust me that analogy is pretty interesting
So I know all of you must have guessed it so these are two beer mugs and all of you who love beard can actually relate to this analogy a lot and I know most of you actually love bears so that’s why I’ve chosen this particular analogy so that all of you
Can relate to it all right jokes apart so fine guys so there’s a beer festival happening near your house and you want to badly go there but your decision actually depends on three factors first is how is the weather whether it is good or bad second is your wife or husband is
Going with you or not and the third one is any public transport is available so on these three factors your decision will depend whether you will go or not so we’ll consider these three factors as inputs to our perceptron and we’ll consider our decision of going or not
Going to the Beer Festival as our output so let us move forward with that so the first input is how the weather will consider it as X1 so when weather is good it’ll be 1 and when it is bad it’ll be zero similarly your wife is
Going with you or not so that would be your X2 if she is going then it’s one if she’s not going then it’s zero similarly for public transport if it is available then it is one else it is zero so these are the three inputs that I’m talking
About let’s see the output so output would be one when you are going to the beer festival and output will be zero when you want to relax at home you want to have bear at home only you don’t want to go outside so these are the two outputs whether you’re going or you’re
Not going now what a human brain does over here okay fine I need to go to the Beer Festival but there are three things that I need to consider but will I give importance to all these factors equally definitely not there’ll be certain factors which will be of high priority
For me I’ll focus on those factors more whereas few factors won’t affect that much to me all right so let’s prioritize our inputs or factors so here our most important factor is weather so if weather is good I love beer so much that I don’t care even if my wife is going
With me or not or if there is a public transport available so I love beer that much that if weather is good then definitely I’m going there that means when X1 is high output will be definitely higher so how we do that how we actually prioritize our factors or
How we actually give importance more to a particular input and less to another input in a perception or in a neuron so we do that by using weights so we assign High wage to the more important factors or more important inputs and we assign low weights to those particular inputs
Which are not that important for us so let’s assign weights guys so weight W1 is associated with input X1 W2 with X2 and similarly W3 with X3 now as I’ve told you earlier as well that weather is a very important factor so I’ll assign a pretty high way to weather analog keep
It at six similarly W2 and W3 are not that important so I’ll keep it as 2 2. after that I’ve defined a threshold value as 5 which means that when the weighted sum of my input is greater than 5 then only my neuron will fire or you
Can say then only I’ll be going to the BF Festival all right so I’ll use my pen and we’ll see what happens when weather is good so when weather is good our X1 is 1 our weight is 6 will multiply it with 6. then if my wife decides that she is going to
Stay at home and she will probably be busy with cooking and she doesn’t want to drink beer with me so she’s not coming so that input becomes zero 0 into 2 will actually make no difference because it will be 0 only then again there is no public transport available
Also then also this will be 0 into 2. so what output I get here I get here as 6. and notice the threshold value it is 5. so definitely 6 is greater than 5. that means my output will be 1 or you can say my neuron will
Fire or I’ll actually go to the Beer Festival so even if these two inputs are zero for me that means my wife is not willing to go with me and there is no public transport available but weather is good which has very high weight value and it
Actually matters a lot to me so if that is high it doesn’t really matter whether the two inputs are high or not I will definitely go to the BF Festival all right now I’ll explain you a different scenario so over here our threshold was five but what if I change this threshold
To three so in that scenario even if my weather is not good uh I’ll give it a zero so zero into six but my wife and public transport both are available all right so 1 into 2 class one into two which is equal to 4. and it is definitely greater than 3.
Then also my output will be one that means I will definitely go to the Beer Festival even if weather is bad and my neuron will fire so these are the two scenarios that I have discussed with you all right so there can be many other ways in which you can actually assign
Weight to your problem or to your learning algorithm so these are the two ways in which you can assign ways and prioritize your inputs or factors on which your output will depend so obviously or in real life all the inputs or all the factors are not as important
For you so you actually prioritize them and how you do that in perception you provide High weight to it this is just an analogy so that you can relate to a perceptron to a real life we’ll actually discuss the math behind it later in the
Session as to how a network or a neuron learns all right so how the weights are actually updated and how the output is changing that all those things will be discussing later in this session but my aim is to make you understand that you can actually relate to a real life
Problem with that of a perceptron all right and in real life problems are not that easy they are very very complex problems that we actually faced so in order to solve those problems a single neuron is definitely not enough so we need networks of neuron and that’s where
Artificial neural network or you can say multi-layer perceptron comes into the picture now let us discuss that multi-layer perceptron or artificial neural network so this is how an artificial neural network actually looks like so over here we have multiple neurons in present in different layers the first layer is always your input
Layer this is where you’re actually feed in all of your inputs then we have the first hidden layer then we have second hidden layer and then we have the output layer although the number of hidden layers depend on your application on what are you working what is your
Problem so that actually determines how many hidden layers you’ll have so let me explain you what is actually happening here so you providing some input to the first layer which is nothing but your input layer you provide inputs to these neurons all right and after some function the output of these neurons
Will become the input to the next layer which is nothing but your hidden layer one then these hidden layers also have various neurons these neurons will have different activation functions so they’ll perform their own function on the inputs that it receives from the previous layer and then the output of
This layer will be the input to the next hidden layer which is hidden layer 2. similarly the output of this hidden layer will be the input to the output layer and finally we get the output so this is how basically an artificial neural network looks like now let me
Explain you this with an example so here I’ll take an example of image recognition using neural networks so over here what happens we feed in a lot of images to our input layer now this input layer will actually detect the patterns of local contrast and then
We’ll feed that to the next layer which is hidden layer 1. so in this hidden layer 1 the phase features will be recognized we’ll recognize eyes nose ears things like that and then that will be again fed as input to the next hidden layer and in this hidden layer we’ll
Assemble those features and we’ll try to make a face and then we’ll get the output that is the face will be recognized properly so if you notice here with every layer we are trying to get a more abstract version or the generalized version of the input so this
Is now basically an artificial neural network how it works all right and there’s a lot of training and learning which is involved that I’ll show you now training on neural network so how we actually train our neural network so basically the most common algorithm for training a network is called back propagation
So what happens in back propagation after the weighted sum of inputs and passing through an activation function and getting the output we compare that output to the actual output that we already know we figure out how much is the difference we calculate the error and based on that error what we do we
Propagate backwards and we’ll see what happens when we change the weight will the error decrease or will it increase and if it increases when it increases by increasing the value of the variables or while decreasing the value of variables so we kind of calculate all those things
And we update our variables in such a way that our error becomes minimum and it takes a lot of iterations trust me guys it takes a lot of iterations we get output a lot of times and then we compare it with the model with the actual output then again we propagate
Backwards we change the variables then again we calculate the output We compare it again with the desired output of the actual output then again we propagate backwards so this process keeps on repeating until we get the minimum value all right so there’s an example that is
There in front of a screen don’t be scared of the terms that I use I’ll actually explain with an example so this is the example over here we have 0 1 and 2 as inputs and our desired output or the output that we already know is 0 1
And 4. all right so over here we can actually figure out that desired output is nothing but twice of your input but I’m training a computer to do that right the computer is not a human so what happens I actually initialize my weight I keep the value as 3. so the
Model output will be 3 into 0 is 0 3 into 1 is 3 3 into 2 is 6. now obviously it is not equal to your desired output so we check the error now the error that we have got here is 0 1 and 2 which is
Nothing but your difference so 0 minus 0 is 0 3 minus 2 is 1 6 minus 4 is 2. now this is called an absolute error after squaring this error we get square error which is nothing but 0 1 and 4 all right so now what we need to do we need to
Update the variables we have seen that the output that we got is actually different from the desired output so we need to update the value of the weight so instead of 3 our computer makes it as four after making the value as 4 we get
The model output as 0 4 and 8 and then we saw that the error has actually increased instead of decreasing the error has increased so after updating the variable the error has increased so you can see that square error is now 0 4
And 16 and earlier it was 0 1 and 4 that means we cannot not increase the weight value right now but if we decrease that make it as 2 we get the output which is actually equal to desired out but is it always the case that we need to only
Decrease the weight definitely not so in this particular scenario whenever I’m increasing the weight error is increasing and when I’m decreasing the weight error is decreasing but as I’ve told you earlier as well this is not the case every time sometimes you need to increase the weight as well so how we
Determine that all right fine guys this is how basically a computer decide whether it has to increase the weight or decrease away so what happens here this is a graph of square error versus weight so what here what happens suppose your squarer is somewhere here and your computer it starts increasing the weight
In order to reduce the square error and it notices that whenever it increases the weight square error is actually decreasing so it’ll keep on increasing until the square error reaches a minimum value and after that when it tries to still increase the weight the square error will increase so at that time our
Network will recognize that whenever it is increasing the weight off at this point error is increasing so therefore it will stop right there and that will be our weight value similarly there can be one more scenario suppose if we increase the weight but then also the square error is increasing so at that
Time we cannot increase the weight at that time computer will realize okay fine whenever I’m increasing the weight the square error is increasing so it’ll go in the opposite direction so it’ll start decreasing the weight and it’ll keep on doing that until the square error becomes minimum and the moment it
Decreases more the squares again increases so our network will know that whenever it decreases the weight value the square error is increasing so that point will be our final weight value so guys this is what basically back propagation in a nutshell is fine so will move forward and now is the correct
Time to understand how to implement the use case that I was talking about at the beginning that is how to determine whether a node is fake or real so for that I’ll open my pie charm this is my pie charm again guys let me just close this all right
So this is the code that I’ve written in order to implement the use case so over here what we do we import the first important libraries which are required matplotlab is used for visualization tensorflow we know in order to implement the neural networks numpy Ferraris partners for reading the data set
Similarly sklearn for label encoding as well as for shopping and also to split the data set into training and testing tasks all right fine guys so we’ll Begin by first reading the data set as I’ve told you earlier as well when I was explaining the steps so what I’ll do
I’ll use Partners in order to read the CSV file which has the data set after that I’ll Define features and labels so X will be my feature and Y will contain my label so basically X includes all the columns apart from the last column which is the fifth one and because the
Indexing starts from zero that’s why we have written 0 till 4th so it won’t include the fourth column all right and so our last column will actually be our label then what we need to do we need to encode the dependent variable so the dependent variable as I’ve told you
Earlier as well is nothing but your label so I’ve discussed in tutorial you can go through it and you can actually get to know why and how we do that then what we have done we have read the data set then what we need to do is to split
Our data set into training and test it and these are all optional steps you can print the shape of your training and test data if you don’t want to do it yourself fine then we have defined learning rate so learning rate is actually the steps in which the weights
Will be updated all right so that is what basically learning rate is then when we talk about epochs means iterations then we have defined cost history that will be an empty numpy array and its shape will be one and it will include the flow type objects then we have
Defined end dim which is nothing but your X shape of axis 1 which means your column then we’ll print that after that we are defined the number of classes so there can be really two class whether the node can be fake or it can be real
And this model path I have given in order to save my model so I’ve just given a path where I need to save it so I’ll just save it here only in the current working directory now is the time to actually Define our neural network so we’ll first make sure that we
Have defined the important parameters like hidden layers number of neurons in Hidden layers so I’ll take 10 neurons in every hidden layer and I’m taking four layers like that then X will be my placeholder and the shape of this particular placeholder is none comma n underscore dim and underscore dim value
I’ll get it from here and none can be at any value I’ll Define one variable W and I’ll initialize it with zeros and this will be the shape of my weight similarly for bias as well this will be the particular shape and there will be one more placeholder Y dash which will
Actually be used in order to provide us with the actual output of the model there will be one model output and there’ll be one actual output which we use in order to calculate the difference right so we’ll feed in the actual values of the labels in this particular placeholder Y dash
And now we’ll Define the model so over here we have a name the function as multi-layer perceptron and in it we’ll first Define the first layer so the first hidden layer and we are going to name it as layer underscore one which will be nothing but the matrix
Multiplication of X and weights of H1 that is the hidden layer 1. and that will be added to your biases B1 after that we’ll pass it through a sigmoid activation function similarly in layer 2 as well matrix multiplication of layer 1 and weights of H2 so if you can notice
Layer 1 was the network layer just before the layer 2 right so the output of this layer 1 will become input to layer 2. and that’s why we have written layer underscore one it’ll be multiplied by weight H2 and then we’ll add it with the bias similarly for this particular
Hidden layer as well and this particular layer as well but over here we are going to use the railway activation function instead of sigmoid then we are going to define the weights and biases so this is how we basically Define weights is how we basically Define weight so weights H1
Will be a variable which will be a truncated normal with the shape of n underscore dim and underscore hidden underscore one so these are nothing but your shapes all right and after that what we have done we have defined biases as well then we need to initialize all
The variables so all these things actually have discussed in brief when I was talking about tensorflow so you can go through tensorflow tutorial at any point of time if you have any question we have discussed everything there since in tensorflow we need to initialize the variables before we use it so that’s how
We do it we first initialize it and then we need to run it that’s when your variables will be initialized after that we are going to create a saver object and then finally I’m going to call my model and then comes that part where the training happens cos function cause
Function is nothing but you can say an error that will be calculated between the actual output and the model output all right so Y is nothing but our model output and Y dash is nothing but actual output or the output that we already know all right and then we are going to use
The gradient descent Optimizer to reduce air then we are going to create a session object as well and finally what we are going to do we are going to run the session so this is how we basically do that for every Epoch we will be calculating the change in the error as
Well as the accuracy that comes after every Epoch on the training data after we have calculated the accuracy on the training data we are going to plot it for every Epoch how the accuracy is and after plotting that we are going to print the final accuracy which will be
On our test data so using the same model we’ll make prediction on the test data and after that we are going to print the final accuracy and the mean squared error so let’s go ahead and execute this guys all right so training is done and this
Is the graph we have got for accuracy versus Epoch this is accuracy y-axis represents accuracy whereas this is a box we have taken 100 epochs and our accuracy has reached somewhere around 99 so with every Epoch it is actually increasing apart from a couple of instances it is actually keep on
Increasing so the more data you train your model on it will be more accurate let me just close it so now the model has also been saved where I wanted it to be this is my final test accuracy and this is the mean squared error all right
So these are the files that will appear once you save your model these are the four files that I’ve highlighted now what we need to do is restore this particular model and I’ve explained this in detail how to restore a model that you have already saved so over here what
I’ll do I’ll take some random range I’ve taken it actually from 754 to 768. so all the values in the row of 754 and 768 will be fed to our model and our model will make prediction on that so let us go ahead and run this
So when I’m restoring my model it seems that my model is 100 accurate for the values that I’ve fed in so whatever values that I have actually given as input to my model it has correctly identified its class whether it’s a fake node or a real node because 0 stands for
Fake note and one stands for real node okay so original class is nothing but which is there in my data set so it is 0 already and what prediction my model has made is zero that means it is fake so accuracy becomes hundred percent similarly for other values as well
Fine guys so this is how we basically implement the use case that we saw in the beginning so in the slide you can notice that I’ve listed down only two applications although there are many more so neural networks in medicine artificial neural networks are currently a very hot
Research area in medicine and it is believed that they will receive extensive application to biomedical systems in the next few years and currently the research is mostly on modeling parts of human body and a recognizing diseases from various cats for example it can be cardiograms cash cans ultrasonic scans Etc and currently
The research is going mostly on two major areas first is modeling and diagnosing the cardiovascular system so neural networks are used experimentally to model the human cardiovascular system diagnosis can be achieved by building a model of the cardiovascular system of an individual and comparing it with the real-time physiological measurements taken from
The patient and trust me guys if this routine is carried out regularly potential harmful medical conditions can be detected at an early stage and thus make the process of combating disease much easier apart from that it is currently being used in electronic noses as well electronic noses have several potential applications in telemedicine
Now let me just give you an introduction to telemedicine telemedicine is a practice of medicine over long distance via a communication link so what the electronic noses will do they would identify odors in the remote surgical environment these identified odors would then be electronically transmitted to another site where an door generation
System would recreate them because the sense of the smell can be an important sense to the surgeon tele smell would enhance telepresent surgery so these are the two ways in which you can use it in medicine you can use it in business as well guys so business is
Basically a diverted field with several general areas of specialization such as accounting or financial analysis almost any neural network application would fit into one business area or financial analysis now there is some potential for using neural networks or business purposes including resource allocation and scheduling I have listed down two
Major areas where it can be used one is marketing so there is a marketing application which has been integrated with a neural network system the airline marketing tactation is a computer system made of various intelligent Technologies including export systems a feed forward neural network is integrated with the
AMT which is nothing but Airline marketing tactation and was trained using back propagation to assist the marketing control of airline seat allocation so it has wide applications in marketing as well now the second area is credit evaluation now I’ll give you an example here the hnc company has developed
Several neural network applications and one of them is the credit scoring system which increases the profitability of existing model up to 27 so these are few applications that I’m telling you guys neural network is actually the future people are talking about neural networks everywhere especially after the
Introduction of gpus and the amount of data that we have now neural network is actually spreading like play right now Why can’t we use feed forward Networks now let us take an example of a feed forward Network that is used for image classification so we have trained this particular Network for classifying various images of animals now if you feed in an image of a dog it’ll identify
That image and will provide a relevant label to that particular image similarly if you feed in an image of an elephant it will provide a relevant label to that particular image as well now if you notice the new output that we have got that is classifying an elephant has no
Relation with the previous output that is of a DOT or you can say that the output at time T is independent of output at time T minus 1. as we can see that there is no relation between the new output and the previous output so we can say that in feed forward networks
Outputs are independent to each other now there are few scenarios where we actually need the previous output to get the new output let us discuss one search scenario now what happens when you read a book you’ll understand that book only on the understanding of your previous
Words all right so if I use a feed forward Network and try to predict the next word in a sentence I can’t do that why can’t I do that because my output will actually depend on the previous outputs but in the feed forward Network my new output is independent of the
Previous outputs that is output at t plus 1 has no relation with output at T minus 2 T minus 1 and a t so basically we cannot use feed forward networks for predicting the next word in a sentence similarly you can think of many other examples where we need the previous
Output some information from the previous output so as to infer the new output this is just one small example there are many other examples that you can think of so we’ll move forward and understand how we can solve this particular problem so over here what we
Have done we have input at T minus 1 will feed it to our Network then we’ll get the output at T minus one then at the next time stamp that is at time T we have input at time T that will be given to our Network along with the
Information from the previous time step that is T minus 1 and that will help us to get the output at T similarly our output for t plus 1 we have two inputs one is a new input that we give another is the information coming from the
Previous times tabs that is T in order to get the output at time t plus one similarly it can go on so over here I have just written a generalized way to represent it there’s a loop where the information from the previous time stamp is flowing and this is how we can solve
This particular challenge now let us understand what exactly our current neural networks neural network I’ll take an analogy suppose your Gym trainer has made a schedule for you the exercises are repeated after every third day now this is the order of your exercises first day you’ll be doing shoulder secondary
You’ll be doing biceps third day you’ll be doing cardio and all these exercises are repeated in a proper order now what happens when we use a feed forward Network for predicting the exercise today so we’ll provide in the inputs such as day of the week month of the
Year and health status all right and we need to train our model or a network on the exercises that we have done in the past after that there’ll be a complex voting procedure involved that will predict the exercise for us and that procedure won’t be that accurate so
Whatever output will get would be as accurate as we want it to be now what if I change my inputs and I make my inputs as what exercise I’ve done yesterday so if I’ve done shoulder then definitely today I’ll be doing biceps similarly if I’ve done biceps yesterday today I’ll be
Doing cardio similarly if I’ve done cardio yesterday today I’ll be doing shoulder now there can be one scenario where you are unable to go to gym for one day due to some personal reasons you could not go to the gym now what will happen at that time we go one time stamp
Back and will feed in what exercise that happened day before yesterday so if the exercise that happened day before yesterday was shoulder then yesterday there were biceps exercises all right similarly biceps happened day before yesterday then yesterday would have been cardio exercises similarly if cardio would have happened day before yesterday
Yesterday would have been shoulder exercises all right and this prediction the prediction for the exercise that happened yesterday will be fed back to our Network and these predictions will be used as inputs in order to predict what exercise will happen today similarly if you have missed your gym
Say for two days three days or one week so you need to roll back you need to go to the last day when you went to the gym you need to figure out what exercise you did on that day feed that as an input and then only you’ll be getting the
Relevant output as to what exercise will happen today now what I’ll do I’ll convert these things into a vector now what is a vector vector is nothing but a list of numbers all right so this is the new information guys along with the information from the prediction at the
Previous time step so we need both of these in order to get the prediction at time T imagine if I have done shoulder exercises yesterday so this will be one this will be 0 this will be 0. now the prediction that will happen will be biceps exercise because if I have done
Shoulder yesterday to relatably biceps so my output will be 0 1 and 0 and this is how vectors work guys so I hope you have understood this guys now this is how a neural network looks like guys we have new information along with the information from the previous time stamp
The output that we have got in the previous time step will use certain information from that will feed into our Network as inputs and then that will help us to get the new output similarly this new output that we have got will take some information from that feed in
As an input to our Network along with the new information to get the new prediction and this process keeps on repeating now let me show you the math behind the recurrent neural networks so this is the structure of a recurrent neural network guys let me explain you
What happens here now consider a Time T equals to 0 we have input X naught and we need to figure out what is h naught so according to this equation h of 0 is equal to Wi weight Matrix multiplied by our input X of 0 plus WR into h of 0
Minus 1 which is H of minus 1 and time can never be negative so we this particular equation cannot be applied here plus a bias so w i into X of 0 plus b h passes through a function G of H to get H of 0 over here after that I want
To calculate y naught so for y naught I’ll multiply h of 0 with the weight Matrix wi and I’ll add a bias to it and pass it through a function G of I to get y naught now in the next timestamp that is at time T equals to 1 Things become a
Bit tricky now let me explain you what happens here so at time T equals to 1 I have input X1 I need to figure out what is H1 so for that I’ll use this equation so I’ll multiply wi that is weight matrix by the input X1 plus W R into h
Of 1 minus 1 which is 0 h of 0 we know what we got from here so w r into h of 0 plus the bias pass it through a function G of H to get the output as H1 now this H1 will use to get y1 we’ll multiply H1
With Wy plus a bias and we’ll pass it through a function G of Y to get y1 similarly the next time stamp that is at time T equals to 2 we have input X2 we need to figure out what will be H2 so we’ll multiply the weight Matrix wi with
X of 2 plus WR into h of 1 that we have got here plus b of H and pass it through a function G of H to get H of 2. from h of 2 will calculate y of 2 w y into h of
2 plus b y that is the bias pass it through a function G of Y to get white and this is how recurrent neural network works guys now you must be thinking how to train a recurrent neural network neural network uses back propagation algorithm for training but back
Propagation happens for every time stamp that is why it is commonly called as back propagation through type and I’ve discussed back propagation in detail in artificial neural network tutorial so you can go through that over here I won’t be discussing back propagation in detail I’ll just give you a brief
Introduction of what it is now with bar propagation there are certain issues namely Vanishing and exploding radians let us see those one by one so in Vanishing gradient what happens when you use back propagation you tend to calculate the error which is nothing but the actual output that you already
Know minus the model output output that you got through your model and the square of that so you figure out the error with that error what do you do you tend to find out the change in error with respect to change in weight or any variable so we’ll call it weight here so
Change of error with respect to weight multiplied by learning rate will give you the change in weight then you need to add that change in weight to the old way to get the new weight all right so obviously what we are trying to do we are trying to reduce the error so for
That we need to figure out what will be the change in error if my variables are changed right so that way we can get the change in the variable and add it to our old variable to get the new variable now over here what can happen if the value d
E by D W that is a gradient or you can say the rate of change of error with respect to our variable weight becomes very small than one like it is 0.00 something so if you you multiply that with the learning rate which is definitely smaller than one then you get
The change of weight which is negligible all right so there might be certain examples where you know you are trying to predict say a next word in a sentence and the sentence is pretty long for example if I say I went to France dash dash dash I went to France then there
Are certain words then I say few of them speak Dash now I need to predict speak what will come after speak so for that I need to go back in time and check what was the context which will be very complex and due to that there’ll be a
Lot of iterations and because of that this error this change in weight will become very small very small so the new way that will get will be actually almost equal to your old weight so there won’t be any updation of way that should be happening and that is nothing but
Your Vanishing gradient all right I’ll repeat it once more so what happens in back propagation you first calculate the error this error is nothing but the difference between the actual output the amount model output and the square of that with that error we figure out what
Will be the change in error when we change a particular variable save weight so d e by DW multiply it with learning rate to get the change in the variable or change in the weight now we’ll add that change in the way to our old weight
To get the new weight this is what by propagation is guys all right I’m just giving you a small introduction to back propagation now consider a scenario where you need to predict the next word in the sentence and your sentence is something like this I have been to
France then there are a lot of words after that few people speak and then you need to predict what comes after speak now if I need to do that I need to go back and understand the context what is it talking about and that is nothing but your long-term dependencies so what
Happens during long term dependencies if this d e by DW becomes very small then when you multiply it with n which is again smaller than one you get Delta W which will be very very small that will be negligible so the new way that you will get here will be almost equal to
Your old weight so I hope you’re getting my point so this new weight so there will be no updation of wage guys this new weight will definitely be will always be almost equal to our old weight so there won’t be any learning here so that is nothing but your Vanishing
Gradient problem similarly when I talk about exploding gradient it is just the opposite of Vanishing gradient so what happens when your gradient or d e by DW becomes very large becomes greater than greater than one all right and you have some long term dependencies so at that
Time your D by DW will keep on increasing Delta W will become large and because of that your weights the new weight with that will come will be very different from your old weight so these two are the problems with back propagation now let us see how to solve these problems
Now exploding gradients can be solved with the help of truncated BTD back propagation through time so instead of starting back propagation at the last time stamp we can choose a smaller time stamp like 10 or we can clip the gradients at a threshold so that it can
Be a threshold value where we can you know clip the gradients and we can adjust the learning rate as well now for Vanishing gradient we can use a radioactivation function similarly we can also use lstm and grus now let us understand what exactly are lsts so guys we saw what are the two
Limitations with the recurrent neural networks now we’ll understand how we can solve that with the help of lstms now what are lstms long short term memory networks usually called as lstms are nothing but a special kind of recurrent neural network and these are current neural networks are capable of learning
Long-term dependencies now what are long-term dependencies I’ve discussed at the previous slide but I’ll just explain it to you here as well now what happens sometimes we only need to look at the recent information to perform the present task now let me give you an example consider a language model trying
To predict the next word based on the previous ones if we are trying to predict the last word in the sentence say the clouds are in the sky so we don’t need any further context it’s pretty obvious that the next word is going to be Sky now in such cases where
The gap between the relevant information and the place that it’s needed is small rnns can learn to use the past information and at that time there would be such problems like Vanishing and exploring gradient but there are few cases where we need more context consider trying to predict the last word
In the text I grew up in France then there are some words after that comes I speak fluent French now recent information suggests that the next word is probably the name of a language but if we want to narrow down which language we need the context of France from
Further back and it’s entirely possible for the gap between the relevant information and the point where it is needed to become very large and this is nothing but long-term dependencies and the lstms are capable of handling such long-term dependencies now lstms also have a chain-like structure like recurrent neural networks now all the
Recurrent neural networks have the form of a chain of repeating modules of neural networks now in standard rnns the repeating module will have a very simple structure such as a single tan H layer that you can see now this standards layer is nothing but a squashing function now what I mean by squashing
Function is to convert my values between -1 and 1. all right that’s why we use talent and this is an example of an RNN now we’ll understand what exactly are LS teams now this is a structure of an LST or if you notice lstm also have a chain-like structure but the repeating
Module has different structures instead of having a signal neural network here there are four interacting in a very special way now the key to lstm is the cell State now this particular line that I’m highlighting this is what what is called the cell State the horizontal line running through the top of the
Diagram so this is nothing but your cell State now you can consider the cell State as a kind of a conveyor belt it runs straight down the entire chain with only some minor linear interactions now what I’ll do I’ll give you a walkthrough of lstm Step by Step all right so we’ll
Start with the first step all right guys so the first step in our lstm is to decide what information we are going to throw away from the cell State and you know what is the sales state right I’ve discussed in the previous slide now this decision is made
By the sigmoid layer so the layer that I’m highlighting with my cursor it is a sigmoid layer called the forget gate layer it looks at h t minus 1 that is the information from the previous Times tab and x t which is the new input and outputs a number between zeros and one
For each number in the cell State CT minus 1 which is coming from the previous timestamp uh one represents completely keep this while a zero represents completely get rid of this now if we go back to our example of a language model trying to predict the
Next word based on all the previous ones in such a problem the cell State might include the gender of the present subject so that the correct pronouns can be used when we see a new subject we want to forget the gender of the old subject right we want to use the gender
Of the new subject so we’ll forget the gender of the previous subject here this is example to explain you what is happening here now let me explain you the equations which I’ve written here so ft will be combining with the cell State later on that I’ll tell you so currently
Ft will be nothing but the weight Matrix multiplied by HT minus 1 and x t and plus the bias and this equation is passed through a sigmoid layer all right and we get an output that is 0 and 1 0 means completely uh get rid of this and
One means completely keep this all right so this is what basically is happening in the first step now let us see what happens in The Next Step so the next step is to decide what information we are going to store in the previous step we decided what information we are going
To keep but here we are going to decide what information we are going to store here all right what new information we are going to store in the cell State now this has two parts first a sigmoid layer this is called a sigmoid layer and which
Is also known as an input gate layer decide which values we’ll update all right so what values we need to update then there’s also a tanash layer that creates a Vector of the candidate values c bar of T minus 1 that will be added to
The state later on all right so let me explain it to you in a simpler terms so whatever input that we are getting from the previous timestamp and the new input it will be passed through an sigmoid function which will give us I of T all
Right and this I of T will be multiplied by c t which is nothing but the input coming from the previous time stamp and the new input with that is passed through a tan H that will result in C T and this will be later added on to our
Sales State and the next step will combine these two to update the states now let me explain the equations so I of T will be what weight Matrix and then we have HT minus 1 comma x t multiplied by the weight Matrix plus the bias pass it
Through a sigmoid function we get I oft c bar of T will get by passing a weight Matrix HD minus 1 x t plus bias through a Tana squashing function and we’ll get c bar of T all right so as I’ve told you earlier The Next Step will combine these
Two to update the state let us see how we do that so now is the time to update the oh wholesale State CT minus 1 with the new cell State CT all right and the previous steps we have already decided what to do we just need to actually do
It so what we’ll do we’ll multiply the old cell State CT minus 1 with f t that we got in the first step forgetting the things that we decided to forget earlier in the first step if you can recall then what we do we add it to it and CT then
We add it by the term that will come after multiplication of i t and c bar T and this new candidate value scaled by how much we decided to update each state value all right so in the case of the language model that we were discussing
This is where we would actually drop the information about the old subject’s gender and add the new information as we decided in the previous steps so I hope you are able to follow me guys all right so let us move forward and we’ll see what is the next step now our last step
Is to decide what we are going to output and this output will depend on US Cell state but it will be a filtered version now finally what we need to do is we need to decide what we are going to output and this output will be based on
Our cell State first we need to pass HT minus 1 and X3 through a sigmoid activation function so that we get a output that is OT all right and this OT will be in turn multiplied by the cell state after passing it through an edge squashing function or an activation
Function and why we do that just to push the values between -1 and 1. so after multiplying OT that is this value and a tan h c t will get the output H2 which will be our new output and that will only output the part that we decided to whatever we have
Decided in the previous steps it will only output that value all right now I’ll take the example of that language model again since I just saw a subject it might want to Output information relevant to a verb and in case that’s what is coming next for example it might
Output whether the subject is singular or plural so that we know what form of a verb should be conjugated into all right and you can see from the uh you can see the equations as well again we have a sigmoid function then that uh whatever output we get from there we multiply it
With tan h c t to get the new output all right guys so this is basically uh lstms in a nutshell so in the first step we decided what we need to forget in the next step we decided what are we going to add to our cell State what new
Information we are going to add to our sales State and we were taking an example of the gender throughout this whole process all right and in the third step what we do we actually combined it to get the new cell State now in the first step what we did we finally got
The output that we want and how we did that just by passing HT minus 1 and XT through a sigmoid function multiplying it with the tan h c t the tanh chart new cell State and we get the new output fine guys so this is what basically lstm
Is guys now we’ll look at a use case where we be using lstm to predict the next word in a sentence all right let me show you how we are going to do that so this is what we are trying to do in our use case guys we’ll feed ilstm with
Correct sequences from the text of three symbols for example had a general and a label that is Council in this particular example eventually our network will learn to predict the next symbol correctly so obviously we need to train it on something let us see what we are
Going to train it on so we’ll be training ilstm to predict the next word using a sample short story that you can see over here all right so it has basically 112 unique symbols so even comma and full stop are considered as symbols all right so this is what we are
Going to train it on so technically we know that lstms can only understand real numbers all right so what we need to do is we need to convert these unique symbols into a unique integer value based on the frequency of our currents and like that we’ll create a dictionary for example we
Have Hader that will have value 20. a will have value 6 General will have value 33 all right and then what happens our lstm will create a 112 element Vector that will contain the probability of each of these words or each of these unique integer values all right so since
0.6 has the highest probability in this particular Vector it will pick the index value of 0.6 then it will see it what symbol is attached to that particular integer value so 37 is attached to counter so this will be our prediction which is absolutely correct as a label
Is also Council according to our training data all right so this is what we are going to do in our use case so guys this is what we’ll be doing in our today’s use case now I’ll quickly open my pycharm and I’ll show you how you can
Implement it using python we’ll be using tensorflow which is a popular python library for implementing deep neural networks or neural networks in general all right so I’ll quickly open my pycharm now so guys this is my buy charm and I over here I’ve already written the
Code in order to execute the use case that we have so first we need to do is import the libraries uh numpy Ferraris tensorflow we know tensorflow dot contract from that we need to import RNN it’s in random collections in time all right so this particular block of code
Is used to evaluate the time taken for the training after that we have log underscore path and this log underscore path is basically telling us the path where the graph will be stored all right so there will be a graph that will be created and then that graph will be
Launched then only our RNN model will be executed then that’s how tensorflow work guys so that graph will be created in this particular path all right and we are using summary writer so that will actually create the log file that will be used in order to display the graph
Using 10 support all right so then we have defined uh training underscore file which will have our stories on which will train our model on then what we need to do is read this file so how are we going to do that first is read line
By line whatever content that we have in our file then we are going to strip it that means we are going to remove the first and the last white space then again we are splitting it just to remove all the white spaces that are there after that we’re creating an array and
Then we are reshaping it now during the reshape if you notice this minus one value tells us the compatibility all right so when you’re reshaping it you need to make sure that uh you know we are providing in the correct parameters to reshape it so you can convert a three
Cross two Matrix to a two cross three Matrix right so just to make sure that it is compatible enough we add this minus one and it will be done automatically all right then return content after that what we are doing we are feeding in the training data that we
Have training underscore file we are feeding in our story and calling the function read underscore data then what we are doing we are creating a dictionary what is the dictionary we all know key value pairs based on the frequency of occurrences page symbol all right so from here collections dot
Counterwords dot most common so most common words with the frequency of occurrence will be a dictionary created and after that we’ll call this dict function and this dict function will feed in word and which is equal to length of dictionary that means whatever the length of that particular dictionary
Is how many time it is repeated so we’ll have the frequency as well as a symbol that will be our key value pair and we are reversing it as well then what we are doing we are calling it uh build underscore data set and we’re feeding in
Our training data there this is our vocabulary size which is nothing but the length of your dictionary and then we have to find various parameters such as learning rate uh iterations or epochs then we have display step and underscore input now learning rate we all know what
It is uh the steps in which our variables are updated training underscore iterations is nothing but your epox the total number of iterations so we have given 50 000 iterations here then we have display underscore step that is thousand which is basically a batch size so batch size is what after
Every thousand epochs you’ll see the output all right so it will be processing it in patches of thousand iterations then we have n underscore input S3 now the number of units in the RNN cell will keep it as 512. then we need to Define X and Y so X will be our
Placeholder that will have the input values and Y will have all the labels all right over cap size so X is a placeholder where we’ll be feeding in our input dictionary similarly Y is also one word placeholder and it’ll have a shape of none comma vocab size vocab size we have defined
Earlier as you can see which is nothing but the length of your dictionary then we are defining weights as well as biases after that we have defined our model all right so this is how we are going to Define it we’ll create a function RNN when we’ll have X weights
And biases and after that we are calling in RNN dot multi RNN cell function and this is basically to create a two-layer lstm and each layer has an underscore hidden units after that what we are doing we are generating the predictions but once we have generated the prediction there are n underscore input
Outputs but we only want the last output all right so for that we have written this particular line and then finally we are making a prediction we are calling this RNN function feeding in X weights and biases after that we are calculating the loss as and then we are optimizing
It for calculating the Ross we are using uh reduce underscore means soft Max cross entropy and this will give us basically the probability of each symbol and then we are optimizing it using RMS prop Optimizer all right and this gives actually a better accuracy than Adam him
Either and that’s the reason why we are using it then we are going to calculate the accuracy and after that we are going to initialize the variables that we have used as we have seen in tensorflow that you need to initialize all the variables unlike constants and placeholders in
Tensorflow all right and once we are done with that we are feeding in our values then calculating the accuracy how accurate it is and then when optimization is done we are calculating the elapsed time as well so that will give us how much time it took in order
To train our model then this is just to run the tensorboard on a local host 6006 and yeah and this particular block of code is is used in order to handle the exceptions so exceptions can be like whatever word that we are putting in might not be there in our dictionary or
Might not be there in our trailing data so those exceptions will be handled here and if it is not there in our dictionary then it will print word not in a dictionary all right so fine guys let’s uh input some values and we’ll have some
Fun with this mod all right so the first thing that I’m gonna feed in is had a general so whenever I feed in these three values had a general there will be a story that will be generated by feeding back the predicted output as the next symbol in the inputs all right so
When I feed in harder General so it will predict the correct output as Council and this Council will be fed back as a part of the new input and our new input will be a general counsel so it will be a general counsel all right so these
Three words will become a new input to predict the new output which is two all right and so on so surprisingly lstm actually creates a story that you know somehow makes sense so let’s just read it add a general counselor to consider what measures they could take to outwit
Their common enemy The Cat by this means we should always know when she was about and could easily all right so somehow it actually makes sense when you feed in that so what will happen when you feed in these three inputs it will predict the next word that is Council after that
It will take counsel and it will feedback as an input along with a general so a general counsel will be your next input to predict two similarly in the next iteration it will take general counsel 2 and predict consider for us and this will keep on repeating foreign Based deep learning framework which is actually the high level API of tensorflow and now I have four major highlights for you guys so let’s check it out from the start well kiras basically runs on top of thiano tensorflow or cntk since it runs on top of any of these Frameworks Keras is
Amazingly simple to work with you might be wondering why well building models are as simple as stacking layers and later connecting these graphs guys kiras attracts a lot of attention but why since it is open source it is actively developed by all the contributors across the world and the documentation is
Nearly endless while we’re good with the documentation but how does Keras perform guys since it is an API which is actually used to specify and train differentiable programs high performance naturally follows through here so now that we know what Keras actually is who makes Keras What It Is Well we need to
Check out some of the contributors and the backers to the Deep learning framework well guys kiras had over 4800 contributors during its law launch and the initial stages and now that number has gone up to 250 000 active developers well what amuses me is that there is a
2X growth ever since every year since its launch also it holds really good amount of traction among multiple startups big players like Microsoft Google Nvidia and Amazon actively contribute to the development of Keras good enough so at this point I’m sure all of you are curious about who uses
Kiras well kiras has an amazing industry traction and it is used in the development of popular firms like Netflix Uber Expedia help and more well now that you know the next time you watch a movie on Netflix or book an Uber you know Keras is being used so guys if
Keras is getting all this attention what makes it so special what makes kiras Stand Out Among all the top framework here I have for you top 10 Snippets that makes Keras so special well guys the focus on user experience has always been the major part of Keras
And next larger option in the industry definitely we just checked out all of the industry attraction it gets and this holds well and next it is multi backend and supports multi-platform as well this helps all the coders come together and code easily next up the research Community present for kiras is amazing
Along with the production community so this is a win-win for me guys so what do you think and moving on all the concepts are really easy to grasp with kiras guys and next up it supports fast processing which is really good and guys it runs seamlessly on both the CPU and the GPU
And it has support for both Nvidia and AMD as well and the best part for me is the freedom to design on any architecture and then later implement it as an API for your projects guys this definitely is a major advantage for me so next up for all the beginners Keras
Is really simple to get started with and I’m here to help you guys with this tutorial for that so stay tuned and lastly the easy production of models makes Keras that special guys now that we know kiras is special guys let’s dig in a little bit about one of
The major Concepts which make Keras what it is the user experience well in my opinion this is very important for anyone who wants to know more about Keras or better they want to start creating their own neural Nets using Keras so clearly kiras is an API designed for humans well why so because
It follows the best practices for reducing cognitive load which ensures that the models are consistent and the corresponding apis are simple and moving on kiras provides clear feedback upon occurrence of any error and this minimizes the number of user actions required for the majority of the common use cases guys
And lastly kiras provides High flexibility to all of its developers well we all love High flexibility right so how is Keras doing this guys it’s very simple it integrates with lower level deep learning framework languages like tensorflow or theano so guys this ensures that you can Implement anything
In kiras which you actually built in your base language which is amazing so next up we need to talk about how Keras supports the claim of being able to support multi-platform and lets us work with multiple back-ends you can develop Keras in python as well as R the code
Can be run with tensorflow cntk piano or mxnet totally based on your requirement this almost feels like a tailor-made API for the framework guys the code can be run on the CPU or the GPU as well support for both the big players being Nvidia and AMD here so this ensures that
Producing models with Keras is really simple total support to run with tensorflow serving GPU acceleration such as q-da when using modules such as web Keras and keras.js Native support to develop Android and iOS apps using tensorflow and core ML and yes full-blown support to work with
Raspberry Pi as well so guys moving on we need to check one quick concept which forms the backbone as a working principle of Keras so let’s check out a computation graph here is an example for you guys just before decoding and working our way through the graph let’s
Look at the features so guys do note that computational graphs are used for expressing complex expression as a combination of simple operations for grass to work with it is mainly useful for calculating the derivatives during the phase of back propagation and hence it makes it easier to implement
Distributed computing on the whole so all it takes is to specify the inputs outputs and to make sure that the graph is connected throughout I hope you guys know what a connected graph is so let’s check out our graph we’ll be working away from the leaf nodes to the top so
Guys here as you can see equal to C multiplier with d where C is equal to a plus b and d is equal to B plus 1. so all we’re doing is we’re making sure we land at e equal to C cross T which is
The head of our tree and we get to this by performing two operations on the leaf nodes so as you can see walking down further equal to a plus b into B plus 1 actually makes sense now and in our case A and B are inputs so guys it is as
Simple as that so next let’s dive a little bit deeper and check out the two major models guys so the first model is a sequential model the working is basically like a linear stack of layers so the first thing that comes into my mind when I think about
The layered approach is the sequential model it is majorly useful for building simple classification Network and encoder decoder model guys and definitely yes this is the model which we all know and love so here we basically treat every layer as an object that feeds into the next layer and so on
And now in the simple chord we’ll import Keras into python we Define the model as sequential and with the hidden layers we have 20 neurons and we’ll be using relu here relu is rectified linear unit guys it is one of the activation functions we’ll be using well model.fit is used to
Train the network here by Epoch I’m sure all of you guys are familiar with it already so it is basically the forward and the backward pass of all of our training examples and batch size is really straightforward as well it is the number of training examples in one
Forward and backward pass guys so higher the batch size the more the memory you need so next we need to check out the functional model it is widely used and it holds good for about 95 percent of the use cases well imagine the concept of playing with Legos guys I’m pretty
Sure most of us have played with Lagos in our childhood it’s pretty much the same here as well well the highlights of the functional model is that it supports multi-input multi-output and an arbitrary static graph topology we have branches so whenever we have a complex model the model is formed into two or
More branches based on the requirement guys the code which we have here is pretty much similar to the previous one but with subtle changes we first import the models we work on its architecture and lastly we train the network well with functional models we have this concept called as domain adaption so
Guys what we did until this stage is that we train a model on one domain but test it on the other this definitely results in poor performance on the overall test data set because the data is different for each of the domains right so what’s the solution for this
Well we adapt the model to work on both the domains at the same time and guys we’ll be looking at a very interesting use case using the functional models in the upcoming slides so stay tuned for that so moving on we need to understand about the two basic types of execution in
Keras deferred and eager execution it is also called a symbolic and imperative execution as well well with deferred we use Python to build a computation graph first like we previously discussed and then this compile graph gets executed well with eager execution there is a slight change guys it is sure that the
Python run time itself becomes the execution runtime for all of the models it is very similar to execution with numpy so if you’re familiar with numpy then it’s a cakewalk guys so on the whole here is a quick note symbolic tensors don’t have a value in the python
Code as of yet but eager tensors will have a value in the python code and with eager execution we make use of a type of recurrent neural networks called as trees so guys it is basically a value dependent Dynamic topology structure so what are you guys thinking about Keras
At this point well it is actually really easy to grasp guys well next let’s look at the steps needed to implement our own neural network using Keras there are five major apps here guys so starting out we need to prepare the inputs for the model we do this by analyzing our
Requirements and specifying the input Dimensions well as you know it the common inputs are images videos text or audio based on your model requirement the next step is to actually Define the artificial neural network model here we do everything from defining the model architecture to building the computation
Graph and also defining the style we’ll be using for the model it is as straightforward as that well step 3 is to specify the optimizer think of it this way a neural network is just a complex function we need to simplify the process of making the machine learn well
The optimizer is just for that there are many types of optimizers such as SGD which is stochastic gradient descent we have RMS Prof which is based on root mean square and atom and so on and the next step is to define the loss function so for every step in our training we’ll
Be checking the accuracy of prediction by comparing the obtained value with the actual one we check for the difference between them and we print out the loss guys we well the goal is to actually Define a function which we will use to reduce the losses in each pass of the
Training phase there are many types of loss functions such as MSE which is mean squared error and we have cross entropy loss which is also called as the log loss in most cases and so on and the last step is to actually train the network based on the input data which is
Also called as a training data and after training we will need to test the model based on the trained data to check if the model actually learned anything so it is as simple as this guys what do you think I would love to know your views on
This so head to the comment section let’s have an interaction there and now guys let’s spice things up a bit I’m sure you guys were curious about the use case so let me walk you through the entire thing well we’ll be checking out a wine classifier in this use case so
Let’s begin by checking out our problem statement so we’re trying to predict the price of a bottle of wine just by knowing the description and the variety of wine well we can work this out with the Keras functional API and tensorflow we’ll be building a wide and a deep
Network using Keras to make predictions for us well can we achieve this goal yes we can this is a problem statement suited for wide and deep learning networks as I mentioned well it involves textual input and there isn’t any correlation between the wines description and its price well this is
What makes it fun in my opinion guys so next we need to check out the model a lot of Keras models are built using the sequential model API as I told you earlier but let’s make use of the functional API for our use case well true the sequential API is still the
Best way to get started with kiras why because it lets you define models easily as a stack of layers like I explained earlier however the functional model allows for more flexibility and is best suited for models with multiple inputs so we need to know a little bit about
Wide and deep model guys well wide models are models with sparse feature vectors well what I mean by sparse feature vectors is that it consists of mostly zeros and a little bit of ones and deep networks and networks which do really well on tasks like speech and image recognition so now that that’s
Sorted we need to take a look look at the data set well for this case we’ll be using a wine data set from kaggle so what’s the data well it’s basically 12 Columns of data and it’s as follows here we’ll be talking about the country that
The wire is from next up is description a few sentences from the sommelier descripting the weinstased smell look and feel a sommelier is a person who is a professional wine taster guys next up is designation the wine yard within the winery where the graves of the wine has been made from
Next up is points the number of points that the wine enthusiasts rated the wine on a scale of 1 to 10. however they only say that they post reviews for the wines which cause greater than equal to 80. next up is price the cost of the bottle of the wine obviously followed by
Province The Province or the state where the wine is from next up I have something called as region one well with region one it’s the wine growing area and a province or a state let’s say for example India next I have Region 2 well sometimes there are more specific
Regions within a specified wine growing area for example we can say Bangalore India but this value can sometimes be blank as well next up is Taster’s name well as it suggests it’s the name of the person who tasted and reviewed the wine taster Twitter handle Twitter handle for
The person who tasted and reviewed the wine well the title of the wine review which often contains the Vintage if you are interested in extracting that feature variety the type of the grapes that is used to make the wine let’s say for example Pinot Noir that’s a type of
Grape and finally the winery that made the fine guys overall goal here is to actually create a model that can identify the variety Winery and the location of a wine based on the description alone and this data set offers some really great opportunities for sentiment analysis and other tax
Related Predator models as well so now that that’s clear we need to take a look at the sample data case so here we have a description for the wine such as scent if it’s tart firm or needs more decanting Etc so this forms our input for the model guys and the output our
Model provides just from all of this textual information is the pricing that it predicts how cool is that guys so basically we need to check out some of the prerequisites before jumping into the code since you’re working with python we’ll require pandas will require numpy skycat learn and Jupiter notebook
So yes kiras works on top of tensorflow so will require both Keras and tensorflow to be installed on the machine so now that that’s done moving on let’s look at a small piece of code here are all the inputs that we’ll require to build the model and lastly we
Test the presence of tensorflow by printing the installed version well without tensorflow it wouldn’t make any sense so we go to kaggle and download the data and end up converting the data to a pandas data frame guys well that’s good enough to start let’s look at the
Code for this model I’ll quickly open up Google collaboratory which is basically a jupyter notebook hosted on that Google Cloud you can actually do this on your local machine as well by installing all of the Frameworks that I have previously mentioned so let me go ahead and open
Collaboratory and let’s begin guys so guys we’ll be executing each of these blocks and will be going on from there so let us check out the first block so here we import all of the modules that we require so guys let me run it and that’s done so next we need to install
The latest version of tensorflow well with Google collab it doesn’t require any extra setup so it’s pretty much straightforward so guys that took about two minutes and tensorflow is installed and as I explained before we need to import the models that we’ll use to build the model
And after that we’ll actually run the code to check the version output of the tensorflow that we just installed and the output we’re supposed to be expecting is version 1.7 because 1.7 is a tensorflow version that we installed as you can check out the output you have tensorflow version 1.7 so beautiful so
Moving on we need to download the data which is from a CSV file hosted on the cloud so let’s go ahead and do that now that it is downloaded and ready let us convert all the data from the CAC file into a pandas data frame as I
Mentioned so now that that is done let’s go ahead and mix up the data by shuffling it and yes so let’s start printing samples from our data set so we’ll be printing the first five rows and as you can see all the columns that we discussed earlier from country price
Province all the way till a detailed description of the wine is present here so now that that’s done next we need to do some pre-processing to limit the number of varieties of wine in the data set so let us go ahead and set a threshold of 500 in our case so anything
Less than 500 will be removed from analysis in our model we’ll be replacing it with Nan which is not a number instead of letting it to be blank so let me go ahead and run this so now that that’s done the next step we need to do is actually split the data
Into a training data set and the testing data set so let’s go ahead and do that and print the size of both the training data set and the testing data set so let me go ahead and run this code for you guys and there it is we have the
Training data set size and the testing data set size now that we have the size we need to actually extract the testing and the training features and all of the labels so let me go ahead and run the score and we can actually get the labels
And so that does it the training and the testing features and the labels are known to our machine by now so now it’s very obvious that we’re using a test description well instead of looking at every word that we found in our description in our data set let us limit
Our bag of words to let’s say top 12 000 words in our data set so guys don’t worry there is actually a built-in Keras utility for creating Justice vocabulary this is considered as wide because the input to our model for each description will be a 12K element wide Vector with
Zeros and ones indicating the presence of word from our vocabulary in a particular description well kiras has some handy utilities for text pre-processing that we’ll use to convert the text descriptions into a bag of words with the bag of words model we’ll typically want to use only a subset of
All the total words that we found in our data set and in this example I used 12 000 words but this is a hyper parameter that you can tune well you can try out a few values to see what works best on your data set well we can use the Keras
Tokenizer class to create our bag of words vocabulary for us so let me go ahead and run this to create the tokenizer okay so now that that’s done we’ll be actually using text to Matrix function to convert each description to a bag of words vectors so let me go ahead and run it
Well now that that’s done guys in the original kaggle data set there are about 632 total varieties of wine to make it easier for our model to extract the patterns we did a bit of pre-processing to keep only the top 40 varieties well around 65 of the original data set or
96k total examples well we use a Keras utility to convert each of these varieties to integer representation and then we’ll create 40 element wide one hot vectors for each input to indicate the variety so let me go ahead and run it guys so now that that’s run guys at
This stage we are ready to build our wide model well guys Keras has two apis for building the models the sequential API and the functional API the functional API gives us a bit more flexibility in how we Define our layers and lets us combine multiple feature inputs into one
Layer it also makes it easy for our wide and deep models into one when we are ready guys with the functional API we can Define our wired model in just a few lines of code as you see well first we need to Define our input layer as a 12K
Element Vector well for each word in our vocabulary and then we’ll connect this to our dense output layer to generate the price prediction so let me go ahead and run this well now that that’s done we’ll compile the model so that it is ready to use if we were using the wide
Model all on its own this is when we’d actually start training it with the fit function and evaluate later with the evaluate function since we are going to combine it with our deep model later on we can hold off on training until the two models are combined which is done
Later guys so let me go ahead and execute this so we Define our wide model and yup our wide model is done let’s go ahead and print out a summary from the white model well now that we have a summary we can realize the total number of trainable parameters and non-trainable parameters
Well in our case the non-trainable parameters are zero guys so guys that’s the end to the construction of the white model and it’s time to build our deep model so let’s go ahead and check that well to create a deep representation of the wines description we’ll represent it
As an embedding well there are a lot of resources on word embeddings but the short version is that they can provide a map word to vectors so that the similar words are closer together in the vector space well to convert our text descriptions to an embedding layer we
Need to First convert each description to a vector of integers corresponding to each word in our vocabulary we can do that with the handy Keras text to sequence method and now we’ve got the integerized description vectors we need to make sure that they’re all the same to feed into
Our models well kiras is fancy and it has a handy method for that too we’ll use part underscore sequences to add zeros to each description Vector so that they’re all the same length well in this case I use 170 as the max length so no descriptions were cut short
So let me go ahead and run this well with our descriptions converted to vectors that are all the same length we’re ready to create our embedding layer and then feed it into our deep model guys so let’s start building our deep model well there are two ways to create an
Embedding layer we can use weights from the pretend embeddings as I previously explained or we can actually learn the embeddings from our vocabulary well it’s best to experiment with both and to see which performs better on your particular data set here we’ll consider using learned embeddings well firstly we’ll
Define the shape of our input to the Deep model then we’ll feed it into the embedding layer and here I’m using an embedding layer with eight Dimensions well you can experiment this with tweaking the dimensionality of your embedding layer as per your choice and the output of the embedding layer will
Be a three-dimensional vectors with the following shape well it will have a bad size a sequence length well in our case the sequence length is 170 it will have an embedding Dimension it is 8 in our example and in order to connect our embedding layer to the dense fully
Connected output layer we actually need to flatten it first so let’s go ahead and Define our model and then we can flatten our layer well once the embedding layer is flattened it’s ready to be fed into the model and to compile it so let’s go ahead and compile it and now that the
Compilation is done as you can see the loss function we’ve used in this case is MSC which is mean squared error the optimizer we’ll be using is atoms and the matrix’s accuracy and at this point of time we have established the wired model and the Deep model so once we have
Defined both our models combining them is really really easy guys we simply need to create a layer that concatenates the output from each of the model and then merge them into a fully connected dense layer and finally Define a combined model that combines the input and the output from each one well
Obviously since each model is predicting the same thing which is the price the outputs or the labels from each one will be the same also guys do note that since the output of our model is a numeric value we will not need to do any pre-processing and it’s already in the
Right format as well how cool is that well that that’s done guys it’s time for the training and the evaluation well you can experiment with the number of training epochs and the batch size that works best for your data set well this is going to take some time so I’ll save
You the Pain by fast forwarding this a bit so guys as you can see the training is actually done so each of the epoch took about 100 seconds and we had 10 epochs were the same so guys here’s the important thing that you have to notice with every Epoch we were actually
Reducing the loss all the way from 1100 to 130 guys and the accuracy of prediction went from 0.02 all the way till 0.0994 which is almost 0.1 well wow that’s definitely a breakthrough for just 10 passes guys and now that the training is done it’s
Time to evaluate it so let me go ahead and run this piece of code for you guys so that was quick that took only about five seconds and we have evaluated the model and now it’s time for the most important part guys seeing how our model actually performs on the data that it
Has never seen before to do this we’ll actually call the predict function or our train model and we’ll be passing it our data set so let’s go ahead and do just that well now that that’s done we’ll have to compare the predictions to the actual values for the first 15 wines from our
Test data set so guys as you can see we have a set of predictions from the description and the predicted value is about 24 dollars well the actual value is 22 dollars next up we have 34 dollars as a predicted one while the average is
70 well this is not a really good case but okay that’s tolerable and next up we predicted 11.9 when the actual values tell wow that is actually really close so next up we have 15.7 versus 9 well this goes on and on for the first 15 and it’s actually really really good well
Guys pretty well it turns out that there are some relationship between a wines description and its price well we might not be able to see it instinctively but our machine learning models certainly can so lastly let’s compare the average difference between the actual price and the model’s predicted price well the
Average prediction difference is about 10 dollars for every wine bottle wow that is really really nice case What are social government well as the name suggests they help us search for something but what exactly it is a root or a path that we can follow to reach our destination in the most optimal way possible you would have heard about the traveling salesperson problem in some way or the other the
Problem is basically that A salesperson needs to travel between various points in a city so that he can sell everything that he has but he has to keep the cost of his traveling as less as possible before computers all of this was manual and had a lot of time and monetary
Wastage but now that is not the case we have many algorithms that do the work for us all we need to do is to feed them with maps or graphs they process the data obtained through them and output the best possible path for traveling so that is basically what a search
Algorithm is we have many algorithms developed that could match every case such as the digixtra the breadth first search and the depth first search a star aostar and so many more this video focuses mainly on the Easter algorithm because of its features let’s now talk
About it which is also a next topic so what exactly is the a-star algorithm it is an advanced breakfast search algorithm that so searches for shorter Parts first rather than the longer Parts Asta is optimal as well as a complete algorithm what do I mean by Optimal and
Complete optimal meaning that the esta algorithm is sure to find the least cost path from the source to the destination and complete meaning that it is going to find all the paths that are available to us from the source to the destination so that makes a star the best algorithm
Right well in most cases yes but a star is slow and also the space it requires is a lot as it saves all the possible parts that are available to us this makes other faster algorithms have an upper hand over a star but it is nevertheless one of the best algorithms
Out there so why choose a star over other faster algorithms let the graphs below answer that for you I have taken the digixtra and the esta algorithm for comparison you can see here that the digit stress algorithm finds all the paths that can be taken without finding
Or knowing which is the most optimal one for the problem that we are facing this leads to the unoptimized working of the algorithm and unnecessary computations algorithm on the other hand finds the most optimal path that it can take from The Source in reaching the destination
It knows which is the best part that it can take from its current state and how it needs to reach the destination so now that you know why we choose a star let’s understand a bit of theory about it as it is essential to help you understand how this algorithm Works a
Star as we all know by now is used to find the most optimal path from a source to a destination it optimizes the path by calculating the least distance from one node to the other there is one formula that all of you need to remember
As it is the heart and soul of this algorithm f is equal to g plus h remember this by heart if you want to understand the algorithm properly let’s understand what each of these parameters means and what makes them so important f is the parameter of Easter which is
The sum of all the other parameters G and H and is the least caused from one node to the next node this parameter is also responsible for helping us find the most optimal path from our source to the destination G is the cost of moving from one node to the other node this
Parameter changes for every node as we move up to find the most optimal path H is the heuristic or an estimated part between the current node to the destination node this cost is not the actual one but is in reality a guess cost that we can use to find which is
The most optimal path between our source and the destination so once you have understood this formula let me show you a simple example to help you understand how this algorithm Works suppose we have a small graph with the vertices s a b and e where s is the source and E is the
Destination you have to also remember that the cost to enter the source and the destination is always going to be zero that means the cost to enter s and the cost to enter e is always going to be 0 right so the heuristic values are s
Is equal to 5 a is equal to 4 B is equal to 5 and E is equal to zero okay so let’s use the formula and calculate the shortest part from the source to the destination now f is equal to g plus h
Where G is the cost to travel and H is a heuristic value so to reach the source F of s is equal to 0 plus 5 is equal to 5. that’s simple enough to understand for now right so we have entered the source moving on the paths from s are the other
Two vertices so the cost of s2a is 1 plus 4 is equal to 5 and the cost for s to b is 2 plus 5 is equal to 7 so now s2a is the shortest path so we choose s to a moving on from here the parts from
A and B to the destination will be calculated now so the total for the path s a e comes up to 14 and the path for SBE comes up to seven so what happens right now we choose the path SBE as it is the shorter one so after the calculation we have found
That B has given us the least path so we change our least part to spe and have reached our destination that is how we use the formula to find the optimal path so with having the example understood I am pretty sure that you will be okay with the algorithm too let me just
Explain it now do not worry it’s simple words and it is easy to understand what is happening you have two lists here which are the open and the closed lists the open list is the node we are currently visiting and are on the closed list over here is what we have not
Visited but will calculate as you know that esta is a complete algorithm so let me just explain the algorithm for now so we add the start node to the list simple enough so for all the neighboring nodes to that start node we will find out which is the least cost of f okay
So once we have done that we switch over to the close list now for all the nodes that are adjacent to the current node that we are on you have to find if there is a node which is reachable if it is not reachable you have to just ignore it
If a node is reachable then check if it is on the open list if it is not move it to the open list and calculate F of G and H now if a node is on the open list make sure to check if that path is lesser than the path that we are
Currently on if so change over to that path now you stop working when you find the destination or you cannot find the destination going through all the possible points that are given to you in the graph or the map okay so that is basically what the algorithm is and I
Hope it is really really easy to understand for you guys so with that context explained it’ll be easier to understand everything that we are going to do from now onwards okay with the algorithm being done let’s move over to how a star is done practically I’ll be showing you two algorithms with extra
And a star which will help you understand where exactly some algorithms fail and how a star can be helpful in finding the path let’s code so as you can see here this is the first algorithm which is the digixtures algorithm and I have basically given it
A maze over here so let me just explain what’s happening in this algorithm so the algorithm basically takes all the points and Maps it to Infinity okay and then finds all the other lists that are really needed and it makes sure that the distance to reach the source is zero
Obviously because if I am in Bangalore and I want to reach Bangalore it’ll always going to be zero right so let’s do that so next we have q is equal to the list of range of n so for all the you know number of nodes that are
Available so you make that so y q is equal to true you have to just find out about the various parts between one node to the other node so if a path is basically Infinity you just break you just come out of the loop else you go through all the various possible nodes
Okay and you check if there is a noon so as you can see here so you are trying to find out a distance where there is an alternate path which is giving you a lesser distance right so for example if I am at Node 1 and I want to reach node
3 there are two possible Parts I can go from node 2 or I can go directly from Node 1 to node 3. but I am currently at node 2 to node 3 I am taking that route right now and that is giving me a value
Of 6 and if I go from 1 to 3 directly it’s just giving me a value of 3. so basically I’m trying to find out all the possible parts that are available and then what I’m going to do is if there is any path which is you know alternate or
Which is much more lesser than what I’m currently on I’m going to move over to that path okay so that’s what this function is basically going to do and I just return the distance and you know the previous ones so then I have another function which is basically just to you
Know display the solution accordingly and here yes I have two mazes over here so wherever you have a 0 over here right so that means it is a travelable path means you can go through that path but if there is a one over here it basically means that it is an obstruction meaning
That I can go from here I can go from here I can go from here I can go from here but I cannot move ahead of here okay so same thing over here and as you can see in the last row it is completely free so that means I can move over from
Here okay and this is also the same thing wherever I have zeros I can travel through that and if it is a one I cannot travel through that so what’s happening over here is I’m just going to you know give out the solution over here so I
Have given one maze over here and I have given the other maze over here so you will be able to find out the shortest path from here so I have given one graph over here and I have given the other graph over here so it will basically
Give me the shortest part between you know the notes to reach one part to the other so let me just run the program so as you can see over here this is the value that I had obtained for one particular graph and this is the you
Know path to travel from 0 to 5 and then there is another path which is all Infinities so INF is basically all the infinite parts and it is telling me that it is not able to find it it’s because the list indexes is not in the way of an
Integer or a float it is basically in INF formats and it is basically meaning that there is no path from this you know this node to the other node so that is the reason I am getting this error actually okay what’s happening over here let me see what’s it for so whenever
There was a graph right so this is the graph this is the input that I’ve given you it had some of the other path because of which it was able to you know calculate to go from one part to the other whereas digixtures failed for this one because it was not able to
Understand which part should I go through because all ones were over here and it had no idea how to go through it right so that was the reason it failed for that particular graph and let me now show you how this is overcome when I use the a style algorithm okay so basically
I have the main function and I’ve given the same you know graph over here but I’m using list so here I can even use double little still give me the same answer and I have the starting path and I have the ending path and the same
Thing over here which I have done so I hope you have understood the mean function right here so what I’m doing over here is I am basically creating a class node for which you know basically it’s like I have g h and f and accordingly all the other parameters
That are required for the a star I have all of that in that and then these are all my open list close list all the you know initializations that I need to do first and then while I am in the open list I am just going to find out for you
Know what are the basic like elements that I can travel from one list to other and make sure that it is on the open list of the closed list you know according to the algorithm I have gone through over here and then if it is on
The end node it basically means that I have reached my destination over here so I am just going to append it and I’m going to return the path basically so then I’m just checking if there is any other path so this function over here right so I’m just checking for a child
Node so basically what this is meaning is that if from one node I’m able to find another position or another root which is much more smaller than what I’m currently on I will change over to that root okay so that’s what this function is basically for and then I’m just going
To find out everything over here so let me just run the program yes so as you can see this was for the first graph that is path and then path one so to reach from 0 to 0 comma 0 to 5 comma Phi it is using this path for one
Graph that is the path and this is the second path right so it is zero zero one one two two three three four four five five so you can understand that even through digit stress field Asta was able to accomplish that even though it has taken a bit more time and computations
It is able to find out the path from going to one place to the other right so that is the reason a star is such a good algorithm but as you know it is basically very very slow this is just a very small example think about it when
It comes to you know huge huge data sets and huge graphs and huge Maps it’s really slow but nevertheless it is one of the best algorithms out there because it is able to give you the output properly so you can see where digit stress fails simply because it tries to
Find a path and it gets stuck in a loop and cannot identify how to come out of the problem esta was able to do that and that’s a very practical example of where astar wins where other fields right Let us understand how AI has actually shaped the space science so far the extraordinary feat the humankind has achieved with their journey into the space is just a start to an infinite reality an artificial intelligence is the key to finding out the answers to the unknown so first of all let me just
Talk about space exploration so I’m going to talk about a few Feats that a human kind is actually achieved with AI and space exploration has been the ultimate guiding force that drives the Innovation into technological advances to explore the unknown in the outer space and the newly discovered
Kepler-90i that orbits a star is one of the Feats achieved by artificial intelligence the planet was actually discovered through the NASA’s Kepler space telescope by using machine learning and according to a news article from NASA Kepler’s Fourier dataset consists 35 000 possible planetary signals an automated tests and sometimes
Human eyes are used to verify the most promising signals in the data however the VK signal often are missed using these methods the shallow and Vandenberg thought there could be more interesting exoplanet discoveries faintly locking in the data so first what they did was they trained the neural network to identify
Transiting exoplanets using a set of 15 000 previously wetted signals from the Kepler exoplanet catalog and in the test set the neural network correctly identified true planets and false positives 96 percent of the time then with the neural network Having learned to detect the pattern of transiting exoplanet the researchers directed their
Model to search for weaker signals nn670 star system that already had multiple known planets their assumption was that multiple Planet systems would be the best places to look for more exoplanets so basically there was a lot of data and they were able to put machine learning algorithms and neural network to get the
Results that they may not have been possible a few decades ago the space agencies all across the globe have realized the importance and relevance of artificial intelligence in space exploration and we are seeing the results with each passing day talking about the global navigation or the gnss so basically a satellite navigation or
Sat nav system is a system that uses satellites to provide autonomous geospatial positioning it allows small electronic receivers to determine their location to a high Precision using time signals transmitted along a line of site by radio from satellites now to understand how AI can help in global navigation the data collected by the
Global navigation satellite system can be used to detect in real time events like tsunami and other disastrous situations by processing the data collected by the gnss using the artificial intelligence and machine learning algorithms and it can study the parameters like temperature gases and other signals that can define an
Immediate danger although these factors may not look significant enough from the ground but years and years of data from the space can actually detect even the smallest of changes and using the data and other parameters and a few machine learning and artificial intelligence algorithms the experts can figure out
Events that may have been finding like a needle in a haystack and with the gnss it is possible to study a lot of data and using the artificial intelligence it is possible to help us in a lot of ways to study the atmosphere or in case of
Large fires let’s say of RS fire it can also figure out the best possible plan of action as well now talking about the communication part the NASA spacecraft typically rely on human controlled Radio Systems to communicate with the Earth and as collection of space data increases NASA looks to cognitive radio
The infusion of artificial intelligence into space Communications Network to meet the demand and increase efficiency so software defined radios like cognitive Radio use artificial intelligence to imply under utilized portions of the electromagnetic spectrum without the human intervention and these white spaces are currently unused but already licensed segments of the
Spectrum so the FCC permits a cognitive radio to use the frequency while unused by its primary user until a user becomes active again so in the future a NASA cognitive radio could even learn to shut itself down temporarily to mitigate the radiation damage during severe space weather events an Adaptive radio
Software could circumvent the harmful effect of space weather increasing science and exploration data returns and a cognitive radio network could also suggest alternate data paths to the ground and these processes could prioritize the root data through multiple paths simultaneously to avoid interference and the cognitive radios artificial intelligence could also
Allocate ground station downlinks just hours in advance as opposed to weeks and leading to more efficient scheduling so these are a few feeds in the space science led by artificial intelligence among many so let’s take a look at the Future How It actually looks with artificial intelligence for space
Science so the scientists are actually trying to figure out if the artificial intelligence can be used to identify or find asteroids or even Discover Life on nearby planets and the team which included students from France South Africa and United States plus mentors from Academia and from technology company and video developed an algorithm
That could render an asteroid in as little as four days so today the technique is used by astronomers at the reciproc observatory in Puerto Rico to do nearly real-time shape modeling of asteroids along with exoplanetary atmosphere analysis or a couple of FTL examples that show the promise in applying sophisticated
Algorithms to the volumes of data collected by NASA’s more than 100 Missions as NASA heliophysicist notes the space agency gathers about 2 gigabytes of data every 15 seconds from its Fleet of spacecraft but we analyze only a fraction of that data and we have a limited people time and resources
So that is why they actually are looking to utilize these tools more so on this Mission there was a spacecraft that studies the sun’s influence on Earth and near Earth space so back in 2014 just four years after the mission launched there was a sensor which actually stopped returning data
Related to extreme ultraviolet radiation levels and information that is actually correlating with the ballooning of the Earth’s outer atmosphere and there’s affects the longevity of satellites including the International Space Station so computer science doctoral students from Stanford and University of Amsterdam among others when mentors from organizations which included IBM
Lockheed Martin and seti developed a technique that could essentially fill in the missing data from the broken sensor so their computer program could do this by analyzing the data from other sdo instruments along with old data collected by the Broken sensor during the four years that it was actually
Working so to infer that the euv radiation levels that sensors would have detected based on what the other sdo instruments were observing at any given time so they generated basically a virtual sensor using AI and there was a recent development where a 29 year old computer scientist has earned worldwide
Fame for helping develop the algorithm that created the first ever image of a black hole so these are a few Feats and this is how we actually are shaping our future into the space science using AI so what was not actually possible a few years ago now it is actually very much
Within our reach because we have gathered the tools that could actually process the years and years of data that we have collected through all these missions that have been sent out in space like we have sent a spacecraft to Mars and we have a lot of data and we
Could actually make or build some tools and some resources that could actually process all that data and we could actually bring in a lot of information that was not possible on decades ago and one more thing that we are going to see in the future is that there will be a
Lot of role of robots basically you know the AI robots in space science when we are talking about space exploration it is quite obvious that the humans are not basically meant to spend a lot of time in space because of obvious reasons like for the very least there is gravitation
And there are a lot of factors that makes it very difficult for any human being to remain in space for a long time if you be in space for a long time it’ll take toll on your health you will lose a lot of muscle mass and there are a lot
Of things that you have to consider but if you replace a few tasks that a human is present for in the space missions if we replace them with robots that can actually do all the tasks that they are actually doing and we can monitor the whole program the Space Program from
Down Earth using a simple software program not actually a simple program it will be a very sophisticated program an AI program that can remotely access all the tasks that are happening inside the space and we still get the data that we actually need so that is one thing that
We are looking at we actually achieved a lot of features regarding this also like there was a mass mission where we sent Rovers and a Relentless work and dedication with a hint of technology is going to take those space Sciences a very long way and it is quite safe to
Say that a spacewalk wouldn’t be a spectacle after 50 years of time even though it is well within out of our reach right now we might be able to do it for an evening stroll if everything goes as planned maybe a 50 60 years later and now comes the important part
Guys as much as we want to travel to some other planets in the times of Crisis but we actually have to stay here and save our plan first so there is one more thing we can do in these testing times that is to learn our way into the future foreign Computing so cognitive Computing refers to individual technologies that perform specific tasks to facilitate human intelligence basically these are smart decision support systems that we have been working with since the beginning of the internet boom so with recent breakthroughs in technology these support systems simply use better data
Better algorithms in order to get a better analysis of a huge amount of information not just that you can also refer to cognitive Computing as understanding and simulating reasoning understanding and simulating human behavior Now using cognitive Computing systems helps in making better human decisions at work some of the applications of cognitive Computing
Include speech recognition sentiment analysis face detection risk assessment Etc we’ll talk about these in details later so now that you know what is cognitive Computing let’s move on and see how cognitive AI works so cognitive Computing systems synthesize data from various information sources while weighing context and conflicting
Evidence to suggest suitable answers to achieve this cognitive systems include self-learning Technologies using data mining pattern recognition and natural language processing to understand the way the human brain works Now using computer systems to solve problems that are supposed to be done by humans require huge structured and unstructured
Data with time cognitive systems learn to refine the way they identify patterns and the way they process data to become capable of anticipating new problems and moral possible solutions now to achieve these capabilities cognitive Computing systems must have some key attributes first of all it should be adaptive now
Cognitive systems must be flexible enough to understand the changes in the information also these systems must be able to digest Dynamic data in real time and make adjustments as the data and environment change then another attribute is being interactive so human computer interaction is a critical component in cognitive systems users
Must be able to interact with cognitive machines and Define their needs as those needs change the Technologies must also be able to interact with other processors devices and Cloud platforms the next one is iterative and stateful now also these systems must be able to identify Problems by asking questions or
Pulling in additional data if the problem is incomplete the systems do this by maintaining information about similar situations that have previously occurred the next attribute is being contextual now cognitive systems must understand identify and mine contextual data such as syntax time location domain requirements a specific user’s profile
Tasks or goals they may draw on multiple sources of information including structured and unstructured data and visual auditory or sensor data now cognitive Computing is also called as the subset of artificial intelligence there are various similarities and different senses between the two so now let’s move on and understand the
Difference between cognitive Computing and artificial intelligence now the technology is behind cognitive Computing are very similar to the Technologies behind AI these include the machine learning deep learning NLP neural networks Etc but they do have various differences as well now cognitive Computing focuses on mimicking human behavior and reasoning to solve
Complex problems whereas AI augments human thinking to solve complex problems it focuses on providing accurate results while cognitive Computing simulates human thought processes to find solutions to complex problems AI finds patterns to learn or reveal hidden information and find Solutions cognitive Computing also simply supplement information for humans to
Make decisions whereas AI is responsible for making decisions on their own minimizing the role of humans and finally cognitive Computing is used in sectors like customer service Healthcare Industries whereas AI is mostly used in finance security Healthcare retail manufacturing Etc so now that you have an idea about cognitive Computing and
Artificial intelligence combined together known as the cognitive AI let’s understand this in a better way with an example so let’s take this Q’s case now cognitive Computing and Air Technologies that rely on data to make decisions but there are nuances between the two terms which can be found within their purposes
And applications so let us imagine a scenario where a person is deciding on a career change an AI assistant will automatically assess the job Seeker skills find a relevant job where his skills match the position negotiate pay and benefits and at the closing stage it will inform the person that a decision
Has been made on his behalf whereas a cognitive assessment suggests potential career paths to the job Seeker besides Furnishing the person with important details like additional education requirements salary comparison data and open job positions however in this case the final decision must be still taken by the job Seeker now based on these
Scenario we can say that cognitive Computing helps us make smart decisions on our own leveraging machines whereas AI is rooted in the idea that machines can make better decisions on our behalf so these were some of the differences between cognitive Computing and artificial intelligence now let’s move
Ahead and talk about some of the applications of cognitive AI in details and see how together it makes some smart technology and makes it simpler for us so talking about the applications we have the smart iot now this includes connecting and optimizing devices data and the iot but assuming we get more
Sensors and devices the real key is what’s going to connect them then we have the AI enabled cyber security so we can fight these cyber attacks with the use of data security encryption and enhanced situational awareness powered by AI this will provide the document data and network locking using smart
Distributed data secured by an AI key then we also have the content AI so a solution powered by cognitive intelligence continuously learns and reasons and can simultaneously integrate location time of day user habits semantic intensity intent sentiment social media contextual awareness and other personal attributes next up is the
Cognitive analytics in healthcare now the technology implements human-like reasoning software functions that perform deductive inductive and abductive Analysis for Life Sciences applications and finally we have the intent based NLP so cognitive intelligence can help a business become more Analytical in their approach to management and decision making now this
Will work as the next step from machine learning and the future applications of AI will incline towards using this for performing logical reasoning and Analysis so these were some of the common applications of cognitive Ai and also how it is going to change the world of technology and with this we have come
To the end of today’s session and I hope you have understood how this cognitive Computing system is a subset of artificial intelligence and how together both of these can do wonders Thank you now for a robot and environment is a place where it has been put to use now remember this robot is itself the agent for example an automobile Factory where a robot is used to move materials from one place to another now the task we discussed just now have a property in
Common now these tasks involve an environment and expect the agent to learn from the environment now this is where traditional machine learning fails and hence the need for reinforcement learning now it is good to have an established overview of the problem that is to be solved using the Q learning or
The reinforcement learning so it helps to define the main components of a reinforcement learning solution that is the agent environment action rewards and states so let’s suppose we are to build a few autonomous robots for an automobile building Factory now these robots will help the factory Personnel by conveying
Them the necessary paths that they would need in order to build the car now these different paths are located at nine different positions within the factory warehouse the car part include the chassis Wheels dashboard the engine and so on and the factory workers have prioritized the location that contains
The body or the chassis to be the topmost but they have provided the priorities for other locations as well which we will look into the moment now these locations within the factory look somewhat like this so as you can see here we have L1 L to L3 all of these
Stations now one thing you might notice here that there are little obstacle prison in between the locations so L6 is the top priority location that contains the chassis for preparing the car bodies now the task is to enable the robots so that they can find the shortest route
From any given location to another location on their own now the agents in this case are the robots the environment is the automobile factory warehouse so let’s talk about these states so the states are the location in which a particular robot is present in the particular instance of time which will
Denote its States now machines understand numbers rather than letters so let’s map the location codes to number so as you can see here we have map location L1 to the state 0 L to N1 and so on we have L8 as state seven and N line at state 8. next what we’re going
To talk about are the actions so in our example the action will be the direct location that a robot can go from a particular location right consider a robot that is at L2 location and the Direct locations to which it can move are L5 L1 and L3 now the figure here may
Come in handy to visualize this now as you might have already guessed the set of actions here is nothing but the set of all possible states of the robot for each location the set of actions that a robot can take will be different for example the set of actions will change
If the robot is in L1 rather than L2 so if the robot is in L1 it can only go to L4 and L2 directly now that we are done with the states and the actions let’s talk about the rewards so the states are basically 0 1 2 3 4 and the actions are
Also zero one two three four up till eight now the rewards now will be given to a robot if a location which is the state is directly reachable from a particular location so let’s take an example suppose L line is directly reachable from L8 right so if a robot
Goes from LA to n line and vice versa it will be rewarded by one and if a location is not directly reachable from a particular equation we do not give any reward a reward of zero now the reward is just a number here and nothing else
It enables the robots to make sense of the movements helping them in deciding what locations are directly reachable and what are not now with this queue we can construct a reward table which contains all the reward values mapping between all possible States so as you can see here in the table the positions
Which are marked green have a positive revolve and as you can see here we have all the possible rewards that a robot can get by moving in between the different states now comes an interesting decision now remember that the factory administrator prioritized L6 to be the top most so how do we
Incorporate this fact in the above table now this is done by associating the topmost priority location with a very high reward than the usual ones so let’s put 999 in the cell L6 comma L6 now the table of rewards with a higher reward for the topmost location looks something
Like this we have now formally defined all the vital components for the solution we are aiming for the problem discussed now we will shift gears a bit and study some of the fundamental concepts that Prevail in the world of reinforcement learning and queue learning so first of all we’ll start
With the Bellman equation now consider the following square of rooms which is analogous to the actual environment from our original problem but without the barriers now suppose a robot what needs to go to the room marked in the green from its current position a using the specified Direction now how can we
Enable the robot to do this programmatically one idea would be introduce some kind of a footprint which the robot will be able to follow now here a constant value is specified in each of the rooms which will come along the robot’s way if it follows the direction specified above now in this
Way if it starts at location a it will be able to scan through this constant value and will move accordingly but this would only work if the direction is prefix and the robot always starts at the location a now consider the robot starts at this location rather than its
Previous one now the robot now sees Footprints in two different directions it is therefore unable to decide which way to go in order to get the destination which is the Green Room it happens primarily because the robot does not have a way to remember the directions to proceed so our job now is
To enable the what with the memory now this is where the Bellman equation comes into play so as you can see here the main reason of the Bellman equation is to enable the robot with the memory that’s the thing we’re going to use so the equation goes something like this V
Of s gives maximum of a r of s comma a plus comma of v s Dash where s is a particular state which is a room a is the Action Moving between the rooms s Dash is the state to which the robot goes from s and gamma is the discount
Factor now we’ll get into it in a moment and obviously R of s comma a is the reward function which takes a state s and action a and outputs the reward now V of s is the value of being in a particular state which is the footprint
Now we consider all the possible actions and take the one that use the maximum value now there is one constraint however regarding the value footprint that is the room marked in the yellow just below the Green Room it will always have the value of 1 to denote that is one of the
Nearest room adjacent to the green room now this is also to ensure that a robot gets a reward when it goes from a yellow room to The Green Room let’s see how to make sense of the equation which we have here so let’s assume a discount factor
Of 0.9 as remember gamma is the discount value or the discount Factor so let’s take a 0.9 now for the room which is marked just below the one or the yellow room which is the asterisk Mark for this room what will be the V of s that is the
Value of being in a particular state so for this V of s would be something like maximum of a we’ll take 0 which is the initial of r s comma a plus 0.9 which is gamma into one so that gives us 0.9 now here the robot will not get any reward
For going to a state marked in yellow hence the r s comma a is zero here but the robot knows the value of being in the yellow room hence V of s Dash is 1. following this for the other states we should get 0.9 then again if we put 0.9
In this equation we get 0.81 then 0.729 and then we again reach the starting point so this is how the table looks with some value Footprints computed from the Bellman equation now a couple of things to notice here is that the max function helps the robot to always
Choose the state that gives it the maximum value of being in that state now the discount Factor gamma notifies the robot about how far it is from the destination this is typically specified by the developer of the algorithm that would be installed in the robot now the
Other states can also be given their respective values in a similar way so as you can see here the box is adjacent to the green one have one and if we move away from one we get 0.9 0.8101729 and finally we reach 0.66 now the robot now can proceed its way
Through the Green Room utilizing these value Footprints even if it’s dropped at any arbitrary room in the given location now if a robot lands up in the highlighted Sky Blue Area it will still find two options to choose from but eventually either of the parts will be
Good enough for the robot to take because of the way the valley Footprints are not laid out now one thing to note here is that the Bellman equation is one of the key equations in the world of reinforcement learning and Q learning so if you think realistically our
Surroundings do not always work in the way we expect there is always a bit of stochasticity involved in it so this applies to robot as well sometimes it might so happen that the robots Machinery got corrupted sometimes the robot may come across some hindrance on
Its way which may not be known to it beforehand right and sometimes even if the robot knows that it needs to take the right turn it will not so how do we introduce this stochasticity in our case now here comes the mark of decision process now consider the robot is
Currently in the Red Room and it needs to go to the Green Room then let’s now consider the robot has a slight chance of dysfunctioning and might take the left or the right or the bottom turn instead of taking the upper turn in order to get to The Green Room from
Where it is now which is the restaurant now the question is how do we enable the robot to handle this when it is out in the given environment right now this is a situation where the decision making regarding which turn is to be taken is partly random and partly another control
Of the robot now partly random because we are not sure when exactly the robot might dysfunctional and partly under the control of the robot because it is still making a decision of taking a turn right on its own and with the help of the program embedded into it so a Markov
Decision process is a discrete time stochastic Control process it provides a mathematical framework for modeling decision making in situations where the outcomes are partly random and partly under the control of the decision maker now we need to give this concept a mathematical shape most likely an equation which then can be taken further
Now you might be surprised that we can do this with the help of the Bellman equation with a few minor tweaks so if we have a look at the original Bellman equation V of X is equal to maximum of r s comma a plus gamma V of s Dash what
Needs to be changed in the above equation so that we can introduce some amount of Randomness here as long as we are not sure when the robot might not take the expected turn we are then also not sure in which room it might end up in which is nothing but the room it
Moves from its current room at this point according to the equation we are not sure of the S Dash which is the next state or the room but we do know all the probable turns the robot might take now in order to incorporate each of those probabilities into the above equation we
Need to associate a probability with each of the turns to quantify the robot if it has got any explicitness chance of taking this turn now if we do so we get PS is equal to maximum of RS comma a plus gamma into summation of s Dash p s
Comma a comma s Dash into V of s Dash now the p s a n s Dash is the probability of moving from room s to S Dash with the action a and the submission here is the expectation of the situation that the robot incurs which is the randomness now let’s take a
Look at this example here so when we associate the probabilities to each of these terms we essentially mean that there is an 80 chance that the robot will take the upper turn now if you put all the required values in our equation we get V of s is equal to maximum of R
Of s comma e plus comma of 0.8 into V of room up plus 0.1 into V of room down 0.03 into room of V or from left plus 0.03 into V of room right now note that the value Footprints will not change due to the fact that we are incorporating
Stochastically here but this time we will not calculate those value Footprints instead we will let the robot to figure it out now up until this point we have not considered about rewarding the robot for its action of going into a particular room we are only rewarding the robot when it gets to the
Destination now ideally there should be a reward for each action the robot takes to help it better assess the quality of the actions but there was need not to be always be the same but it is much better than having some amount of reward for the actions than having no rewards at
All right and this idea is known as the living penalty in reality the reward system can be very complex and particularly modeling sparse rewards is an active area of research in the domain of reinforcement learning so by now we have got the equation which we have so
What we’re going to do is now transition to Q learning so this equation gives us the value of going to a particular State taking the stochasticity of the environment into account now we have also learned very briefly about the idea of living penalty which deals with associating each move of the robot with
A reward secure learning processes an idea of assessing the quality of an action that is taken to move to a state rather than determining the possible value of the state which is being moved to so earlier we had 0.8 into V of s 1 0.03
Into V of S2 0.1 into V of S3 and so on now if you incorporate the idea of assessing the quality of the action for moving to a certain state so the environment with the agent and the quality of the action will look something like this so instead of 0.8 V
Of S1 will have q of S1 comma A1 will have q of S2 comma A2 Q of S3 now the robot now has four different states to choose from and along with that there are four different actions also for the current state it is in so how do we
Calculate Q of s comma a that is the cumulative quality of the possible actions the robot might take so let’s break it down now from the equation V of s equals maximum of a r s comma a plus comma summation s Dash b s a s Dash into
V of s Dash if we discard the maximum function we have r s of a plus gamma into summation p and v now essentially in the equation that produces V of s we are considering all possible actions and all possible States from the current state that the robot is in and then we
Are taking the maximum value caused by taking a certain action and the equation produces a value footprint which is for just one possible action in fact we can think of it as the quality of the action so Q of s comma a is equal to RS comma a
Plus comma of summation p and v now that we have got an equation to quantify the quality of a particular action we are going to make a little adjustment in the equation we can now say that V of s is the maximum of all the possible values
Of Q of s comma a right so let’s utilize this fact and replace V of s Dash as a function of Q so q s comma a becomes R of s comma a plus gamma of summation p s a s Dash and maximum of the q s Dash a
Dash so the equation of V is now turned into an equation of Q which is the quality but why would we do that now this is done to ease our calculations because now we have only one function Q which is also the core of the dynamic programming language we have only one
Function Q to calculate and R of s comma a is a Quantified metric which produces reward of moving to a certain State now the qualities of the actions are called The Q values and from now on we’ll refer to the value Footprints as the Q values
An important piece of the puzzle is the temporal difference now temporal difference is the component that will help the robot calculate the Q values with respect to the changes in the environment over time so consider our robot is currently in the mark State and it wants to move to the Upper State one
Thing to note that here is that the robot already knows the Q value of making the action that is moving to the Upper State and we know that the environment is stochastic in nature and the reward that the robot will get after moving to the Upper State might be
Different from an earlier observation so how do we capture this change the real difference we calculate the new q s comma a with the same formula and subtract the previously known qsa from it so this will in turn give us the new QA now the equation that we just write
Gives the temporal difference in the Q values which further helps to capture the random changes in the environment which may impose now the name q s comma a is updated as the following security of s comma is equal to Q T minus 1 s
Comma a plus Alpha T DT of a comma s now here Alpha is the learning rate which controls how quickly the robot adapts to the random changes imposed by the environment the QT s comma a is the current state q value and a Q T minus minus comma is the previously recorded Q
Value so if you replace the TDS comma a with its full form equation we should get Q T of s comma a is equal to Q T minus 1 of s comma a plus Alpha into R of s comma a plus comma maximum of q s
Dash a dash minus Q T minus 1 s comma a now that we have all the little pieces of q line together let’s move forward to its implementation path now this is the final equation of Q learning right so let’s see how we can implement this and
Obtain the best path for any robot to take now to implement the algorithm we need to understand the warehouse location and how that can be mapped to different states so let’s start by recollecting the sample environment so as you can see here we have L1 L to L3 till L line and
As you can see here we have certain borders also so first of all let’s map each of the above locations in the warehouse to numbers or the states so that it will ease our calculations right so what I’m going to do is create a new Python 3 file in the jupyter notebook
And I’ll name it as Q learning numpy okay so let’s define the states but before that what we need to do is import numpy because we’re going to use numpy for this purpose and let’s initialize the parameters that is the gamma and Alpha parameters so gamma is 0.75 which is the discount Factor
Whereas Alpha is 0.9 which is the learning rate and that’s what we’re going to do is Define the states and map it to numbers so as I mentioned earlier L1 is 0 and until n line we have Define the states in the numerical form now the next step is to define the
Actions which is as mentioned above represent the transition to the next state so as you can see here we have an array of actions from 0 to 8. now what we’re going to do is Define the reward table so as you can see here it’s the same
Matrix that we created just now that I showed you just now now if you understood it correctly there isn’t any real Barrel limitation as depicted in the image for example the transition L4 to L1 is allowed but the reward will be zero to discourage that path or in tough
Situation what we do is add a minus one there so that it gets a negative reward now in the above code snippet as you can see here we took each of the states and put once in the respective state that are directly reachable from the certain
State now if you refer to that reward table once again Watch we created the above area construction will be easy to understand but one thing to note here is that we did not consider the top priority location L6 yet we would also need an inverse mapping from the states
Back to its original location and it will be cleaner when we reach to the utter depths of the algorithms so for that what we’re going to do is at the inverse map location state to location we will take the distinct State and location and convert it back
Now what we’ll do is we’ll Now define a function get optimal which is the get optimal route which will have a start location and an N location don’t worry the code respect but I’ll explain you each and every bit of the code now the get optimal root function
Will take two arguments the starting location in the warehouse and the end location in the warehouse respectively and it will return the optimal route for reaching the end location from the starting location in the form of an ordered list containing the letters so we’ll start by defining the function by
Initializing the Q values to be all zeros so as you can see here we have given the Q balance to be zero but before that what we need to do is copy the reward Matrix to a new one so this is the rewards new and next again what we’re
Going to do is get the ending State corresponding to the ending location and with this information automatically we’ll set the priority of the given ending state to the highest one right we are not defining it now but will automatically set the priority of the given ending State as 999. so what we’re
Going to do is initialize the Q values to be 0 and in the Q learning process what you can see here we are taking I in range 1000 and we’re going to pick up a state randomly so we’re gonna use the NP dot random Rand int and for traversing
Through the neighbor location in the same maze we’re gonna iterate through the new reward Matrix and get the actions which are greater than zero and after that what we’re going to do is pick an action randomly from the list of the playable actions in years to the next state we’re gonna compute the
Temporal difference which is TD which is the rewards plus gamma into the Q of next state and we’ll take NP dot ARG Max of Q of next State minus Q of the current state we’re gonna then update the Q values using the Bellman equation as you can see here we have the Bellman
Equation and we’re going to update the Q values and after that we’re going to initialize the optimal route with the starting location now here we do not know what the next location yet so initialize it with the value of the starting location which again is the random location now we do
Not know about the exact number of iterations needed to reach to the final location hence why low will be a good choice for the iteration so when you’re going to fetch the starting State fetch the highest Q value of penetrating to the starting State we
Go to the index or the next state but we need the corresponding letter so we’re going to use that state to location function we just mentioned there and after that we’re going to update the starting location for the next iteration and finally we’ll return the root so let’s take the starting location of
N9 and the end location of l y and see what part do we actually get so as you can see here we get L9 L8 L5 L2 and L1 and if you have a look at the image here we have if we start from n line to L1 we
Got L8 L5 L to L1 L8 L5 L to L1 that would yield us the maximum value or the maximum reward for the robot so now we have come to the end of this Q learning session and I hope you got to know what exactly is Q learning with the
Analogy all the way starting from the number of rooms and I hope the example which I took the analogy which I took was good enough for you to understand Q learning I understand the Bellman equation how to make quick changes to the development equation and how to
Create the reward table the queue table and how to update the Q values using the Bellman equation what does alpha do what does gamma do foreign Problem now we know two things we show here that we’ve got water and also we’ve got jug so what is a problem okay so let me just take this problem for you we’ve got two jugs here we name it jug a and jug B for our convenience
And jage can hold four liters of water jug B can hold three liters of water so this is the maximum capacity which we have set now what is the problem the problem here is that how exactly do we get two liters of water in jug a right
So we could simply say that Priyanka why don’t you just pour two letters in this chunky well had it been so easy this would not be a problem in artificial intelligence right so there are more interesting facts to it the thing is that these jugs do not have any markings
On them that means we are working with jugs with no marking or labels and also we do not have any measuring devices now imagine that if we had a jug which had labels and also a measuring device we could easily say that yes this jug is too liters filled or three liters filled
We can easily understand how much quantities there in this jug but we do not have any labels or any measuring devices to measure the quantity in this chart this is what the problem is and we will find various solutions to this problem but before that we will move on
To the importance of water check problem so suppose I start from my house and I want to go to McDonald’s to have a burger so there are several routes that I can take and my natural selection would be the best route that that will help me to reach McDonald’s faster so
Now let us relate this what to the water jet problem now in water jug uh what happens is that we have a start and an end State too like here we have got house we are starting from the house that is our start State and in water jug
We start with empty jugs so and the destination here is the McDonald’s but for a water jug the destination is having two liters of water exactly two liters of water in jug cake this is our gold state so in both the things we can relate that we are having a start State
And a gold state does there are several ways to reach McDonald’s similarly in water jug we have several possible solutions to get exactly two liters of water or arrive at the gold state now if we know how to solve this problem in artificial intelligence where we know
The start State and the goal State this can really help us in getting such States various possible solutions and also the optimal search and what is your problem is the very basic to any problem solving process as it helps us to find various possible ways and also the best
Optimal way so now let’s proceed on to understanding various possible solutions to what it’s a problem but before that there are certain assumptions that we have to keep in mind and that is we can fill a jug from a pump there is Unlimited Supply of water so you don’t
Have to worry about what water also we can pour water out of a jug to the ground so you can also empty the jugs by pouring water out to the ground and another assumption is that we can pour water from one jug to another right you can interchange the water from one jug
To another and another last assumption is that there’s no measuring devices available with us so that we can measure the how much quantity of water is there in the jumps so these are the assumptions which you have to keep in mind before framing any possible solutions in artificial intelligence so
Now let’s move on to devising your Solutions and looking on how artificial intelligence really solves this problem now consider we have two jugs here that we have seen this is a four liter jug and this one I will create it as a three liter jobs capacity and we name
These jars as a and this was named as B right so just for the conveniencing we will refer to this jug as a and this job as B now what we have to do is we have a goal state in mind and that is we want
Two liters capacity in jug a and zero liter capacity in junk B this is our cold state to be reached and our start state is nothing but we start with 0 0 that means assuming that there is no water in these jugs okay so we will
Start with various States and see how we can fill these jugs so now we see that we don’t have any measuring Mark also the entire capacity of this Chuck is 4 liters and this jug is entire capacity is three liters all right so now keeping this in mind we will Design our state
Space so how do we do it let’s just start filling by jug B okay I I filled it full then what I do is I will just transfer whatever I have object B to jug a right now my jug a is three liters full because I have transferred three
Liters from junk B okay now what I do is I will again fill jug B so I have this 3 3 okay I have got Unlimited Supply of water remember the rules the assumptions so I filled this jug B with three liters of water now what I want to do is I can
Fill this jage with one liter more because it has got four liter capacity so what I do is I will transfer certain water from jug B to jug a so this becomes four and since I transfer certain water it became two all right now let’s move on for another thing
Again my goal state is not reached because it is 2 0 and we still have four so now what we have to do I can simply empty the water of jug a to the ground and let it be zero right and let this be 2. now what I can do simple transfer all
The water of jug B to jugging and this is our goal State as simple as that right and this entire thing is known as state space representation now this is only one of the possible solutions that we have now let’s look at another solution how can we reach to the goal
State with starting State as 0 0. now suppose again we have got 0 0 another possible solution could be I fill The Jug a with entire capacity that is four liters now what I do is I empty three letters from jug a and fill jug B with
Three littles okay let me fill this jump B with three letters then what I do is I empty the water of jug B to the ground so it makes it zero now what happens again my goal state is not yet reached right it is 2 here so what I do is I
Will make here 0 and 1. what I’m doing I’m just pouring all the water from so this becomes 0 and this becomes one now again what I do is I will fill my entire jug a with four liters of water and I have here one that is already
Remaining with jug B now what I can do again is that since this jug B has the capacity of three liters it is also having the remaining capacity as two liters more so I I will fill this remaining capacity from jug a so what I
Do is I will fill this so this becomes 2 and this becomes 3. okay so this is again my goal state which has been reached all I was concerned with that I get exactly two liters of water in my jug a right exactly I’m not bothered about how
Much water I have in Chuck B right so these are the two State space representations of filling this or achieving this goal state right so this was how we do it in manually now let us understand how we do it in artificial intelligence so now let’s understand the state space
Representation of the production rules that are artificial intelligence program would understand and based on these rules we’ll also code so understanding this becomes a little important and I’ll make things really simple for you so let’s just stick around and have patience while understanding this so now
One thing we know that our jug a has four liters of capacity and jug B has three liters of capacity right so keeping these two things in mind we will start afresh now what I do is I will take it as a and b okay and this I will
Write here as let’s suppose we will fill it in with 4 comma B that means I fill the entire jug that means the jar here let me have two jars also so I fill the jar a completely with four liters of water and let BBB now here what I do is
There are certain rules which we follow but here we are certain I’ll just write here rules just for our convenience so now what we do again again we’ll write here a b and then let’s fill jar B now these rules are nothing but the possibilities that
We can apply and code right these are the conditions that we can use in our code just like if else conditions okay now another rule is that I would simply write again a b and now coming to another rule that is I withdraw certain amount from jug a so this is simple I
Just withdraw the amount and jug B has the same amount now here the condition is that a is greater than 0 that means there has to be some water in a so that I can withdraw certain water okay now another rule is for B here a would
Remain the same and I will withdraw certain water from B so here the rule applies that we need to have certain motor so B has to be greater than zero only then I can withdraw certain water from B right okay now coming to another rule which is very simple and that is
What is it 0 comma B that means I’m withdrawing or I can just I’m emptying the jar a to the ground completely right so nothing remains in jar a that means that N Jug a so it becomes 0 this is the empty root and again here B should be greater than Z 0.
Again another rule I have is I with empty The Jug B right so here again the condition would be B is greater than zero that has to be something to empty right so that’s what I do it and here it is a not b sorry because I’m emptying
Something from a so it has to be there has to be something here now another rule comes like let’s create another state and what we did was we also pour certain water from A to B and from B to a right so there was some exchange of
Waters between these two jumps so how do how do I code it and how do I uh you know write that in artificial intelligence according so what I do is I would simply just understand this thing my jug a has a capacity of four letters
Right I keep it as it is now what I’m doing is I’m withdrawing certain water from jump B and I’m filling it in jug a right so whatever modification has to be done or whatever things I have to do I have to do it with jug B so I’m taking
Out certain water from bees I’ll use the minus sign and what I’m doing I’m filling it in a so whatever I fill it in a would be something like for the total capacity minus a that means the filled water understand this like suppose This Is
A and B and what I’m doing is this is City this is having three liters capacity I have to you know pour or add certain water from V to a now suppose this is already filled with one liter suppose it is already having it now the remaining capacity becomes how much
Three liters so how much will I write this four minus one that is three so how how can I write it here that means this is I’m withdrawing it from jug B so I will write it as that from V I am withdrawing or I am adding three
Liters of water so this has been written as 4 minus the capacity that is a right okay so similarly I will do this for when the water I have to pour or interchange from how would I do it again I will write here a right and um okay I’ve not talked
About the conditions here so before we proceed let me talk about a little conditions here and these conditions are let me top this out also so that I can write the conditions so the condition for here or the rule here would be nothing but a plus b it should be
Greater than equal to 4 and also one more condition is there that is B should be greater than 0 because I have to give something from B jug B so it has to have certain capacity okay so now coming to another rule and that is now suppose I
Want to withdraw certain water from jug a and I have to fill it in jug B so the condition would be jug a minus entire thing just I’m just reversing this thing the capacity minus the quantity to be added and then I will just write here
The capacity of jug B right and here the condition is nothing but it has to be like a plus b should be greater than equal to 3 and also a should be greater than 0. right now remember we have another condition wherein we just poured
All the water from jug a to jug B right and jug B2 Chucky so for that what we can code is like a plus b and from B that means from jug B becomes zero the water injury so here what we do is we are pouring all the water from chubby to
Jug a and the condition for this would be a plus b it is less than equals to 4 and B should be greater than e 0 because you have to transfer certain water from B to Chuck E now another condition is we’ve made it for like jug in now let’s
Make it for jug B’s so now we empty The Jug a and fill all the water of jug a to jug B and here the condition would be what a plus b should be less than equals to 3 and also a should be greater than 0 because we have to fill certain water
From A to B so these rules as we have seen these are nothing but these are called the production rules and these rules have been followed by the AI programming or this third space to find the best or optimal State spaces for a problem right and how to reach to the
Goal state which was 2 or 0 right and which we have figured out previously that we are reaching it by two possible solutions right so now let us just apply these rules to our previous problems and see how we apply these rules and then we will do on our Hands-On
So now we’ve got these rules listed and let’s just name these rules as you know one so now let’s name these rules as one two okay just for our convenience three four five six and we know what we are doing in each rule there we’re drawing water
Or we are just emptying the jars or we are just refilling it right so we’ve got again for convenience sake let’s just have two jars our previous charge and Juggy has quality capacity jugby has for three liters of capacity so now in this table let’s have a here and let’s
Have B here and let’s just use this and this thing is the basic principle they will be applying in solving the artificial intelligence code also so the same things goes there also so this is a jug a jug B and the rule so now what we did previously was we started with zero
Zero State and that obviously had no rule right and our goal stayed as we can just write it here the goal state is to fill two liters of exact capacity in our jug e right and in jug B we can have 0 either 0 or any any quantity like 2 N
This is a problem known as in right okay so now let us just move on to refilling it so what I do is I will fill the juggly with three liters of water let’s just fill it so we are completely filling it so which rule fills it this
Rule rule two right so let’s just fill rule 2 here right understand now what we are doing is we are simply transferring jug B water to jug a so we’ll just write here three and we write here 0 because this is emptied now since we are
Transferring as you can see here we are transferring this from here from jug B to jug a so this is the ninth rule right right here nine now another thing is that notice that this condition is fulfilled that a plus b should be less than or equal to 4 on B should be
Greater than zero so we can see here you add them 0 plus 3 is 3 which is less than 4 and B is also greater than zero so this condition is met and hence this has been executed so on next all the current status 30 okay so now let’s move
On to our next state what we want now what I do is I just refill my jug B so when I’m refilling it simply with additional water I’m using simple rule that is 2 here right okay now this becomes my current state now let’s move on to another state we’ll repeat until
Our goal state is released that is 2 0. so now let us move on to now what I do to the next state is that I fill certain Border in jug a till its maximum capacity and I transfer some water from jug B so I transferred how much one
Liter is remaining here to make it four and remaining is two here so what what rule am I using here simply we applied this rule here seventh rule right I’m I’m pouring certain water V minus 4 minus a so this rule applies here so I write here 7 right now this becomes my
Current state again my goal is not reached so what I do is I can simply empty jug a and I write here jug B same this rule is simply rule six because I’m simply emptying this thing I’m using these rules it’s not important that I will use all these rules but only the
Rules which are required here to reach the goal state so this is one of the possibility now what I do is I will simply empty this jug B water to juggle so here we get 2 0 and this rule is nothing but ninth root here I am
Emptying this water to here right so this is how I will apply these rules to solve one problem now this is one possible solution another possible solution could be something different wherein we started with you know four and zero so what can that be if I can
Just write here oblique and if you don’t get confused we can simply play it around here we start with 0 0 we start with State four and here B is zero the rule applies is one similarly here what I do is I write here one because I’m
Transferring some of the water to jug B just focus on the oblique sign and this is what this is the second combination I’m doing replicating the same one you can always tell you this and the rules will change right when I’m just changing the state I am just changing the rule
That’s all 0 and 1 and again I will change the rule and that is 10th rule here so depending on the state your rules will change and also the goal State the goal State here is different here we get 2 3 and here we are getting
Two zero now depending on the goal State we will have we will be devising the solutions so these are the two possible solutions that we have obtained by using these rules right so this is how we imply this in artificial intelligence and now let’s see the code in Python by applying these in
Conditions so these rules are very important once we execute these small problems okay so now let’s move on and execute this code in Python all right so now we are here with the water jug problem and the simplest way to implement this is Define a function so
We have defined a function called pore water here and we have taken two our jug a and jagpi and then we have defined the maximum capacities of both these jugs and here we are taken as five and seven let’s increase it and see different solutions for this and fill is four so
You can change the minimum and the maximum capacity is also depending on the goal State you want to attain so now what I do is when I run this code I’ve used if and L if and else control loops here so this would be the
Flow of the code and how it will look we can see when I run it I’ve given the condition with jug B if it is still and it will return the maximum capacity again I am withdrawing certain water I am filling certain water I am pouring
The water from one jug to another jug and this is how the code goes and that’s what we have seen so when you implement this you would get something like this and our goal state was to fill The Jug B with four letters of exact opacity and
Get 0 in Chuck E and that’s what we have gone through and we have started with the start State at 0 0 and the goal set was 0 and 4 to get exactly 4 liters of capacity or four liters of water in jug B okay so this is how the code runs you
Have to apply the logic here like when this iterates it just fills The Jug jug a and then it fills the Jack B so this is how you simply implement it in Python also we can apply various algorithms like breadth first search or depth first search to find the best optimal search
Space or the state space for this problem that could be have been a little complex for this tutorial so we are sticking to a simplest solution here wherein we have to find the goal State as 0 4 and starting from zero zero so this was the implementation foreign Ty this has been the buzzword since the day of its release and people are going crazy about everything it can do Microsoft will be integrating chat GPT into teams to automatically take notes and recommend tasks chat GPT has also passed the U.S medical and law exams and it has got a lot of
Doctors lawyers and Engineers concerned about whether it could replace them people want to know if child GPT is the next step towards our evolution by replacing Google and voice assistance like Siri and Alexa but what exactly is chat GPT why is it changing everything now I want you to
Think of child GPT like Siri or Alexa minus the voice capabilities chat CPT can give you detailed and contextual answers in a very human-like manner it can remember conversations do math write essays and much more and it does it so well that people are genuinely scared of losing their jobs
Now chat GPT is not really A New Concept Microsoft and other companies have also tried this before but for nowhere near successful openai developed a model called gpt3 using huge data sets that had a variety of information they released it to the public calling it playground where a lot of developers
Used it for the daily tasks chat CPT has been another implementation of this opening year it took a year to make this model faster and more accessible to the general public and when they released it in November 2022 the crowd went nuts the side gain more than a million users
In just five days and to give you some perspective this number is bigger than Netflix Twitter Facebook and even Instagram all that said I want to let you know how everyone is using chat GPT to simplify business operations for developers chat GPT has been a blessing since chat GPT can write code
Provide code templates and fix errors most of the problems that developers usually run into has been fixed as a result of this productivity has greatly increased and companies are getting more results is the same thing with content development videos like this one traditionally take a lot of time and effort to be made
We have to think about how best to explain things so that you the viewer can easily understand it while making sure that the content is SEO friendly and extremely engaging chat CPT has also helped us give you better more optimized content while reducing our workload by the way did you
Know that chat GPT can explain stuff better than most University professors I guess this time we contemplated our learning methods marketing and sales has also been much easier than before think about it like this if you want to make a customized sales pitch all you
Need to do is enter the details chat epd can provide a customized sales pitch for each individual lead how convenient is that and it doesn’t just end there chat GPT can also be integrated with accounting and data analytics platforms so you don’t need to know the formulas
Directly all you need to have is data and you just need to type what you want to do if all this doesn’t blow your mind just yet then know that psychologists and psychiatrists are using chat EPT to help and counsel their patients now if you are questioning whether it
Would really be helpful then let me tell you it can I just had a 30-minute conversation with Chad GPT about my dog and it seemed way more interested in him than most of the people I usually talk to about this all that said if you want to start using
Chat GPT then all you need to do is open chat Dot openai.com this is the URL of chat GPT and the moment you open this it will ask you to create an account once you do that you will land in a web page similar to this let’s start with a simple question about
Programming this is just a random question on stack Overflow which I will be using for this demonstration how to check if an array includes a value in JavaScript the user has also provided some code and he has said that he only knows how to do it like this is there a better way
And once we ask chat GPT the same thing boom chat chipity is on it it finds a solution and provides the best answer it could think of if you look at the question while the solution might be simple the complexity of conditions and following my instructions properly is quite hard
Chat GPT is through it and also provided a sample snippet which I can directly copy and paste into my editor but this is not the only thing it can also give you complete guides on how to do something like for example let me type using python help me fetch data for
Nifty bank for the past three years chat GPT is already providing explanation and code of how this can be done and is pretty damn cool now remember that chat EBT remembers conversations and can have future answers based on it so now if I type great how can I create
An AI model that can predict values using this data it immediately understands what I want and provides an accurate answer the answer might not be everything that you hope for but at least now you know where to start from another use case for chat GPT that we
Talked about was content creation to do this let’s create a new chat thread this may show that the previous conversations don’t affect the current conversation if we start without a new chat threat chat GPT might think that we are trying to write content about stocks and machine
Learning we don’t want to do that now do we so let me ask chat GPT to give me blog ideas for chat GPT itself and we already have multiple blog ideas for chat CPT we can then ask it to write a story based on some option
We can then say I like the third option could you create a storyline for it and we can see that child CPT immediately starts generating a storyline based on my option it includes an introduction and sections of what my blog will contain similarly we can also create a sales pitch
Let’s just say create a motorbike sales pitch for a person with the following details and then I have listed down a few details like name product Age Country and source and once we submit this prompt we can see that chat gbt immediately starts writing a beautiful sales pitch you can
Even use the sales patch directly if you want now if we dive too deep into the subject then this video will take forever to finish so if you are interested in knowing more about how to use chat GPT then check a course on chat GPT which
Will not only cover the basics of charge DPT but also the advanced and complex usage for different scenarios but before we end this video I definitely need to make you aware of a few limitations the first thing is that it may occasionally provide incorrect information you should know that chat GPT is a
Relatively new application and it needs some time to improve the current version of this is a free research preview meaning that they have released it to test the application this statement could also mean that we might see an improved paid version of this in the future there’s a lot of
Rumor going about that already the second limitation is that it may occasionally produce harmful or biased content this is something that we have seen in all chatbots in the past and charged GPT has reduced the occurrence but it’s still not perfect but the biggest drawback is that the
Model was trained on data collected before 2021 this means that it has very limited knowledge about the current affairs so make sure to check the facts before drawing conclusions and with all this I want to know what you think of chat GPT how do you think
It will look in the future and what do you think about the future of Automation and AI I personally think that we can expect radical changes in all segments imagine Voice assistance being as intelligent as chat GPT and if it is updated with real-time information things would be much more convenient
Than now you could shop online with chat gpt’s recommendations find good restaurants meet new people plan trips and much more Before we dive into Ai and culture let’s play a little game I like games and I hope you do as well so the name of the game is guess the title of the movie so I’m gonna show you four different movie posters and your job is to guess the
Name of the movie I hope that was clear the instructions were clear guess the name of the movie as you see the posters you can pause the screen and then go down to the comment section below and leave your answers there so are you ready three two one so I hope that you
Left your answers in the comments and I’m gonna give you the titles of course so there they are on the screen the point of this game is to see how many of you recognize the movies and I think most of my viewers will at least recognize two out of the lot yeah and
All of these movies are based on artificial intelligence and if you haven’t watched any of these movies I recommend that you find some time for them they’re mind-blowing The Matrix is my personal favorite from the ones you see on the screen but why did we play
This game well this is the easiest way to show you guys that people get excited when they hear the term AI thrown about because it is cool and it’s everywhere and there’s so much to talk about people have so many different opinions and perspectives on the matter so I think it
Is safe to agree that it’s deeply embedded in not just the Western culture but cultures all over the world wouldn’t it be cool to look at some of the factors or elements that led to such a widespread adoption of artificial intelligence in cultures well first up we’re going to take a look at
Advancement in science movies like the one you see on the screen came out before World War one and they inspired a whole generation of scientists who did some amazing work in the field of computer science and AI so what were the advancements that led to AI the first
Computer was made in 1946 and was called eniac it was enormous it occupied a space of 50 by 30 foot but could only do simple calculations but the benefit was that it was reprogrammed programmable following which in 1950 who is regarded as Alan Turing who is regarded as the
Father of modern computer science wrote in his paper how to build intelligent machines and how to test their intelligence but as you know computers weren’t Advanced to hold any instructions or data but touring’s test for intelligence is still used today over a half a century later isn’t that
Amazing next in 1956 at the Dartmouth summer research project on artificial intelligence is where it all began I mean for artificial intelligence as a matter of fact this is where the term artificial intelligence was first used from 1957 to 1974 AI flourish it became the buzzword and also the computers
Could store more information became faster cheaper and more accessible yet they were too slow to exhibit any kind of intelligence remember the first movie in the game played earlier that came out in 1968 since the computers were too slow and needed to catch up the AI Buzz
Started to quiet down and that was until 1997 when IBM supercomputer called Deep Blue Beat Gary Kasparov at chess and mind you Gary was regarded as the Grand Master of Chess here’s a picture of Deep Blue on the right so you get an idea of how compact the computers got by the
Year 1997. so I hope you had as much fun as I had by giving you a little bit of a history on AI so this was the first element advancement in science and technology that influenced the culture around AI today what’s the second one it is books and literature on AI literature
Can range from the ones that deal in fact such as academic scientific or research papers to the ones that are very imaginative and completely fictional Like Comics inspired by the advancement in science and technology written literature where writers take the most complex and evolving ideas and
Present them in a way of stories so common folks can understand and relate to them but the next element that we are about to see does this better than books in my opinion movies and television shows movies and TV shows have a wider audience than books and like I said
Earlier the right ones Inspire the future scientists and the advancements that they make Inspire the next generation of movies so it’s like a psycho you see let’s move on to the next one different events such as conferences seminars webinars trade shows and Expos Awards and competitions workshops and
Others provided a platform to exchange ideas on artificial intelligence and connected like-minded people they range from highly academic events like the one we saw earlier Dartmouth summer project on artificial intelligence to things like Comic-Con where people attend as their favorite characters and meet movie stars next one is mainstream media like
Newspaper news channels and radio across the world they have provided coverage on artificial intelligence and brought on Experts to talk about it the next one is normal everyday conversations yeah if AI is going to be everywhere then it will be part of everyday conversations all of these different elements and not to
Forget social media have made AI incredibly famous in cultures across the world but in the past few years the buzz around AI is back especially with companies like Amazon Google Facebook Apple Tesla and two or three dozen research organizations making incredible progress in this field so naturally that
Will make you wonder what is AI like right now in this section I have another surprise for you but before we get to it let’s check out what are the different types of artificial intelligence so there are three main categories of artificial intelligence a-n-i-a-g-i a and Asi so obviously the first thing
That popped in your mind if you don’t know about these categories is what are they well let’s briefly take a look artificial narrow intelligence is what ani is it is also known as weak intelligence artificial narrow intelligence refers to AI systems that can only perform a specific task on
Their own using human-like capabilities they can learn from past experiences in regards to that specific task even the most complex AI that uses machine learning and deep learning to teach itself falls under a and I this type of artificial intelligence represents all existing AI including even the most
Complicated and capable AI that has ever been created to this date let me give you some examples Google Alexa and Siri voice assistants use AI to detect speech and Carry Out commands today’s security and survey surveillance systems uses facial recognition which is a type of narrow AI social media platforms use it
To learn about preferences and show you ads and content that you will enjoy Ecommerce websites like Amazon use it to learn about your shopping activities where you are located and so much more to recommend similar products it also helps them figure out inventory for warehouses for different locations and their unbelievable two-day delivery
Banking in financial sector use it for fraud activity detection loan approval and so on last but certainly not the least autonomous vehicle use it to navigate the roads on their own so I hope that you got a little bit of an idea of what a n i is let’s move on to
The second category which is Agi and it stands for artificial general intelligence and it’s also known as strong AI so you’re probably wondering what it is first we talked about artificial narrow intelligence now we’re talking about artificial general intelligence does it give you any idea if you’re thinking that this type of
Artificial intelligence is good and general tasks meaning all tasks instead of a specific task like we saw in the previous category a and I you would be right so AGI will be able to better understand the humans it is interacting with by Discerning that needs emotions beliefs and thought processes it will be
Able to learn things and apply a broad range of areas just like human beings can and unlike narrow artificial intelligence right now AGI is the goal of the field of AI a place where AI will become part of the physical world and will navigate it like we do quite a bit
Of leading AI researchers think we’ll get there in a few decades there’s been a lot of research and development happening on AGI by organizations like open AI Deep Mind apprente and many more so I hope you got an idea about this one let’s move on to the last type which is
ASI and it stands for artificial super intelligence judging by the name super you probably already got an idea that this will be the Pinnacle of a i ASI is where machines will become self-aware like us humans and they will be overwhelmingly Superior than humans at everything that is they will have
Greater memory faster data processing and Analysis and decision making capabilities the potential of having such powerful machines at our disposal seems appealing but these machines may also threaten our existence or at the very least our way of life we don’t have any examples of AI thank God for that
Because we aren’t even ready for artificial general intelligence the one before this one okay let’s switch gears and play a little game because the section after this is going to be a little tense so let’s disperse some tension before we get into dangers of AI
The name of the game is guess the type of AI Ani which is artificial narrow intelligence good at specific tasks AGI is more like human capabilities but it is not self-aware and Asi which is artificial super intelligence which is where the machines become self-aware and get way better than human beings so I
Will put up a few pictures or videos and you will have to guess the type of AI and if you want you can comment your answers below let’s see how many of you get them right sounds good okay let’s begin first one here is an autonomous
Vehicle that can drive from point A to point B and even Park itself without a human in it if you guessed a n i you’d be right second one is a Tesla bot the robot will be able to perform basic repetitive tasks with the aim of eliminating the need for people to
Handle dangerous or boring work like getting groceries from Walmart if you guessed AGI you wouldn’t be completely wrong but its capabilities will be limited so it’s a and I these last ones are from Boston Dynamics they’ve been making incredible strides in robots navigating the world and personally the
Robots give me the Goosebumps I mean look at them move but remember even though they can navigate the world and obstacles around them they are not able to understand the humans so take that as a hint if you will so any guesses if you guessed AGI again you’d be wrong the
Point of this game was to show you that everything that we have today is artificial narrow intelligence all three examples that we saw were artificial narrow intelligence but there are a lot of companies that are working on artificial general intelligence and it’s truly it is around the corner right so I
Wanted you guys to watch this two clips before we move on to the next section freaking Goosebumps right I mean imagine one day you go out for a stroll or a jog and you just see this thing just running loose that’s something from your worst nightmare robots taking over the world
That must make you wonder what are the dangers of AI doesn’t it well then let’s talk about its dangers in this next section so what are the near to return dangers of AI and when I say near to midterm I mean from present day to 20
Years in the future well the first one we’re going to take a look at is privacy imbalances of access to information has been exploited in the recent past and is probably being exploited as you watch this video let’s see some of the examples you understand how everyone’s
Privacy is always at stake in 2018 news broke out that a data analytics firm Cambridge analytica had analyzed the psychological and social behavior of users through Facebook’s likes and targeted them with ad campaigns for the 2016 U.S presidential election now imagine being able to influence U.S presidential election that is crazy second example
Example is Clearview face recognition Clearview is a company that created a face recognition system to help police officers identify criminals they claimed that it only used publicly available images on social media platforms like Facebook Instagram to identify criminals but in January 2020 the New York Times reported that in a demo from Clearview
It scraped personal images from Instagram account of the show’s producer next example is deep fake and this is really concerning images and videos that are created using deep learning and contain a real person acting or saying things that they didn’t do or say are called Deep fakes if you use it for
Entertainment purposes deep fakes are fun but people are creating deep fakes for fake news and information and worse deep fake porn and the last example is mass surveillance in China China uses over 200 million surveillance cameras and facial recognition to keep constant watch on their people and also mine
Their behavioral data capture it on the cameras China also implemented a social credit system to rate the trustworthiness of its Citizen and give them ratings accordingly based on their surveillance so on this system if they rate higher they get more benefits and if they rate lower well you’re out of
Luck all of this was done without their knowledge or consent and that is what is most concerning as you can see people’s privacy is a big concern right now and more so in the future years to come next one we are taking a look at is AI producing biases well naturally you’re
Going to say Kevin how does AI produce biases well Ai and machine learning models use parameters and the data I was trained with to make yes or no decisions there have been many examples in which the parameters don’t tell the full story and the labeling of training data could
Be done with some sort of bias so when AI is used for serious tasks like filtering job candidates giving out loans accepting or rejecting Insurance requests or even for medical diagnosis it can have a tremendous impact on somebody’s life here’s just one for example this image shows how one of
These thermometer guns gets classified as a gun when it is held by a person of dark skin and as a monocular when it is held by a person of salmon or white skin this happened in 2015 where Google’s image recognition software made this bias now imagine something like this
Being used by law enforcement in real life scenarios well I don’t even want to think about that the next one is centralization of AI this danger deals with the fact that what if all the AI technological advancements always end up in the hands of a few people or groups
So we are already seeing this now to a certain degree the amount of time that people spend on platforms like Facebook Instagram Twitter Youtube is so ridiculous and we’re constantly giving data to these companies with each interaction they all have gigantic oceans of data on their users which
Along with their AI enables them to keep making advancement in technology staying ahead of the curve and always on the top but it is a very likely scenario that some of these companies will advance to extremely high-tech machines and the rest of the world wouldn’t be able to
Keep up will they then become all-powerful like United States with atomic bombs in World War II or even worse because AI can unlock a whole bunch of threats to humanity of course there’s this problem of Rich getting richer and also there’s this issue of transparency where we don’t know how
Much these organizations know about us and what they can do here’s another scenario what if one of these companies just like in the movie Iron Man where Tony Stark has his fancy high-tech AI powered suits and there’s people that are always trying to steal it so that they can reverse engineer and then
There’s bad people who want to use this technology to bring Mayhem and destruction to the world so it may not end up being that extreme but it is entirely possible that the these companies may have good motive and good intentions towards the world but an entity with a bad intention can steal it
And use it to cause harm to humanity so centralization of AI in the hands of few people or group is a real threat and we need to make sure that we always keep these threats in check so the next danger is loss in jobs it’s also called
AI dislocation use of AI in the workplace is expected to result in the elimination of large number of jobs though AI is expected to create and make better jobs Education and Training will have a crucial role in preventing long-term unemployment but initially it
Will lead to a lot of lost jobs it is a common misbelief that AI dislocation will only hold true for labor to semi-skilled jobs but the truth of the matter is that it has already started displacing even high skill jobs that require Masters and PhD degree let me
Give you an example one such job is is consultants for companies to help them make decisions they are being replaced by Ai and machine learning softwares let’s now talk a little bit about the long-term dangers and by long term I mean 20 to 50 years into the future so
What’s the first danger well the first danger that we will talk about is Safety and Security AI applications that are in physical contact with humans are integrated into human body could pose safety risks as they may be poorly designed and misused or hacked poorly regulated use of AI and weapons could
Lead to loss of human control over dangerous weapons so the first example is neural link brain implants and the second one is autonomous weapon system the second danger which is very real and it could happen and I think it is going to happen at a certain degree at least
Is the transformation of society remember the times when three four generations would live under the same roof and since the Silicon revolution of 28th and 21st Century has broken down families into nuclear units of two to four people on an average today you look around and you’ll see everyone with
Their heads down and eyes glued to their phones most of the people prefer to spend their time on their phone than the world around them and if you want proof of this you only need to look around go to a public space and just observe people with the AI Revolution that we
Are in we will see entire realities unfold in real time in augmented reality and we will be able to interact with an intelligent projection in them that behave like humans also we will be surrounded by robots that human beings will have relations and feelings towards it is quite possible that on this course
We might forget what it is to be a human and that brings us to the last danger that is AI rise to power I know a lot of people think that Singularity or AI rise to power are taking over the world is not a likely scenario but super intelligent AIS with real World
Tractions such as an access to pervasive data centers and autonomous robots could radically alter their environment example harnessing all available solar chemical and nuclear energy if such AI is found uses for free energy that better further their goals than supporting human life human survival would become unlikely so these were some
Of the scenarios and dangers that we need to avoid while we go forward with the AI Revolution so let’s now find out in this last and final section what does the future hold for us and important challenge is to determine who is responsible for damages caused by an AI
Operated device or service in an accident involving a self-driving car for example should the damage be covered by the owner the car manufacturer the programmer the person who trains the machines on data it’s really unclear right now but with that being said the future of AI is very proud I’m saying
It’s very bright and it feels like a start of a revolution world is also taking baby steps towards artificial general intelligence and it is going to be really really helpful for humankind but that also means that we should tread really carefully and we should take all the dangers as very real because this
Isn’t something that human beings could control once it gets out of hand as opposed to every other technology that we had previously okay so how should we prepare ourselves for the future educating ourselves about AI or other Tech is going to be absolutely Paramount if you’re going to make this AI
Revolution of net positive effect for the whole world so for that we need to have tough conversation and debates especially when it comes to developments in artificial general intelligence artificial narrow intelligence is not as concerning as what we are trying to achieve right now and then lastly we
Need to ask difficult questions keep the progress in check by establishing ethics and laws on AI and if required there should be licensing and registration of every single Tech that we make so here’s my closing remark with great power comes great responsibility just as the world thought that the development of nuclear
Weapons will wipe out the entire planet but we now know that if handled with great care responsibility and Universal cooperation then it could not only lead to World Peace but resolve energy crisis of the world with nuclear power So what is knowledge representation actually now knowledge representation in AI describes the representation of any knowledge basically it is a study of how the beliefs intentions and judgments of an intelligent agent can be expressed suitably for automated reasoning now one of the primary purposes of knowledge representation includes modeling intelligent Behavior foreign agent
Knowledge representation and reasoning also known as KR or the krrs represents information from The Real World for a computer to understand and then utilize this knowledge to solve complex real life problems like communicating with human beings in natural language now knowledge representation is AI is not
Just about storing data in a database it allows a machine to learn from that knowledge and behave intelligently like a human being now there are different kinds of knowledge that need to be represented in AI such as the objects events performance facts then we have meta knowledge and the knowledge base
So these were the different kinds of knowledge that you need to represent and now that you know about knowledge representation in AI let’s move on and know about the different types of knowledge here now talking about the different types of knowledge there are five types talking about the first one
We have declarative knowledge now this includes Concepts facts and objects expressed in a declarative sentence then we have the structural knowledge now it is a basic problem solving knowledge that describes the relationship between Concepts and objects next up we have the procedural knowledge now this is responsible for
Knowing how to do something and includes rules strategies procedures Etc the fourth one is the meta knowledge now this defines knowledge about other types of knowledge and finally we have the heuristic knowledge now this one represents some expert knowledge in the field or subject so these were the five
Important types of knowledge in AI now let’s have a look at the cycle of knowledge representation and how it actually works so talking about the cycle of knowledge representation artificial intelligence systems usually consist of various components to display their intelligent Behavior now these components include perception learning knowledge representation and reasoning
Then we have planning and finally execution now here is an example to show the different components of the system and how it works now this diagram shows the interaction of an artificial intelligence system with the real world and the components involved in showing the intelligence so first of all the perception component
Retrieves data or information from the environment now with the help of this component you can retrieve data from the environment find out the source of noises and check if the AI was damaged by anything also it defines how to respond when any sense has been detected the next component is the learning
Component now this learns from the captured data by the perception component here the goal is to build computers that can be taught instead of programming them now learning focuses on the process of self-improvement in order to learn new things the system requires knowledge acquisition inference acquisition of heuristics faster searches Etc
Now moving on to the next components these are the main components in this cycle that is the knowledge representation and reasoning now this shows the human-like intelligence in the machines knowledge representation is all about understanding intelligence instead of trying to understand or build brains from the bottom up its goal is to
Understand and build intelligent behavior from the top down and focus on what an agent needs to know in order to behave intelligently also it defines how automated reasoning procedures can make this knowledge available as needed and finally we have the planning and execution now these components depend on the analysis of knowledge representation
And reasoning here planning includes giving an initial State finding their preconditions and effects and a sequence of actions to achieve a state in which a particular goal holds now once the planning is completed the final stage is the execution of the entire process so this was all about the cycle of
Knowledge representation in artificial intelligence now let’s move on and understand the relationship between knowledge and intelligence so what is this relation between knowledge and intelligence in The Real World Knowledge plays a vital role in intelligence as well as creating artificial intelligence it demonstrates intelligent behavior in AI
Agents or systems now it is possible for an agent or system to act accurately on some input only when it has the knowledge or experience about the input so here is an example to understand this relationship better so here there is one decision maker whose actions are justified by sensing
The environment and using knowledge but if we remove this knowledge part from here it will not be able to display any intelligent Behavior so this is the relationship between a knowledge and intelligence you need the knowledge in order to display any intelligent behavior that is for any intelligent system you need knowledge first
So now that you know the relationship between knowledge and intelligence let’s move on to the techniques of knowledge representation in AI not talking about the techniques there are four techniques of representing knowledge these four techniques include logical representation semantic Network representation frame representation and production rules so talking about the first one logical
Representation is a language with some definite rules which deal with prepositions and has no ambiguity in representation it represents a conclusion based on various conditions and lays down some important communication rules also it consists of precisely defined syntax and semantics which supports the sound inference each sentence can be
Translated into Logics using syntax and semantics so what is the difference between a syntax and a semantic now for syntax it decides how we can construct legal sentences in logic and it determines which symbol we can use in knowledge representation also how to write those particular symbols but when it comes to
Semantics semantics are basically the rules by which we can interpret the sentence in the logic also it assigns a meaning to each of these sentence so let’s talk about some of the advantages and disadvantages of this representation now logical representation helps to perform logical reasoning this representation is also the basis for the
Programming languages these are some of the advantages of logical representation talking about the disadvantages logical representations have some restrictions and are challenging to work with now this technique may not be very natural and inference may not be very efficient now talking about the next Technique we have the semantic Network representation
Now semantic networks work as an alternative of predicate logic for knowledge representation now in semantic networks you can represent your knowledge in the form of graphical networks this network consists of nodes representing objects and arcs which describe the relationship between these objects also it categorizes the object in different forms and links those
Objects now the representation consists of two types of relation first one is an is a relationship which is also known as The Inheritance and then we have another kind of relation now talking about some of the advantages semantic networks are a natural representation of knowledge it also conveys meaning in a transparent
Manner and these networks are simple and very easy to understand but talking about the disadvantages semantic networks take more computational time at runtime and these are inadequate as they do not have any equivalent quantifiers also these networks are not intelligent and depend on the creator of the system
So this was about the semantic Network representation now let’s move on to the next one which is the frame representation now a frame is a record-like structure that consists of a collection of attributes and values to describe an entity in the world now these are the AI data structure that
Divides knowledge into substructures by representing stereotype situations basically it consists of a collection of slots and Slot values of any type and size slots also have names and values which are called the facets moving on to the advantages it makes the programming Easier by grouping the related data frame representation is
Easy to understand and visualize also it is very easy to add slots for new attributes and relations also it is easy to include default data and search for the missing values talking about the disadvantages in frame system inference the mechanism cannot be easily processed the inference mechanism cannot be
Smoothly processed by the frame representation either also it has a very generalized approach now moving on to the final Technique we have the production rules now in production rules agent checks for the condition and if the condition exists then production rules fires and corresponding action is carried out the
Conditioned part of the rule determines which rule may be applied to a problem whereas the action path carries out the associated problem solving steps now this complete process is called a recognize act cycle the production rule system consists of three main parts the first one is definitely the set of
Production rules then we have a working memory and finally the recognize act cycle now talking about the advantages and disadvantages first up is the advantages where the production rules are expressed in natural language and the production rules are highly modular and can be easily removed or modified but talking
About the disadvantages it does not exhibit any learning capabilities and does not store the result of the problem for future users now during the execution of the program many rules may be active thus the rule-based production systems are inefficient so these were the four important techniques for knowledge representation
In AI so now let’s have a look at the requirements for these representations now a good knowledge representation system must have properties like the representational accuracy it should represent all kinds of required knowledge also it must have inferential adequacy it should be able to manipulate the representational structures to
Produce new knowledge corresponding to the existing structure it also must have the inferential efficiency so the ability to direct the inferential knowledge mechanism into the most productive directions by storing appropriate guides then finally it must have the acquisitional efficiency as well that is the ability to acquire new knowledge easily using automatic methods
So these are some of the important requirements for the knowledge representation in AI now let’s understand some of the approaches of this knowledge representation with examples so the first approach is a simple relational knowledge now this is the simplest way of storing facts which uses the relational method here all the facts
About a set of the object are set out systematically in columns also this approach of knowledge representation is famous in database systems where the relationship between different entities is represented now taking an example you can see that we have three different columns like name Asian employee ID and
We have three different names along with their age and employee IDs so this is how you represent a simple relational knowledge it is the simplest way of defining you just have to take the name age and employee ID and Define the relation moving on to the next approach we have the inheritable knowledge
Now in the inheritable knowledge approach all data must be stored into hierarchy of classes and should be arranged in a generalized form or a hierarchical manner also this approach contains inheritable knowledge which shows a relation between instance and class and it is called the instance relation now in this approach objects
And values are represented in boxed nodes so here you can see the example where we have two different players Dany and Peter who plays two different games such as cricket and football but they both are known as players and they both play for the under 19 teams so you can
See the relationship here for Danny cricket and Peter football it’s an instance but when you compare the relationship of cricket football with the player and the under 19 it’s a is a relationship which is also the inheritance now the Final Approach is the inferential knowledge now the inferential knowledge approach
Represents knowledge in the form of formal logic thus it can be used to derive more facts also it guarantees correctness so here if we take an example you can see that we have first statement which is John is a cricketer then we have another statement which says all cricketers are athletes
So you can also represent this as cricketer who is John and give the relationship as cricketers or athletes so here you are checking out the relationship between John cricketer and athletes and also this helps you and guarantees correctness in your relationship so this is the Final Approach which is the inferential
Knowledge and these were some of the important approaches along with their examples that you need to know for knowledge representation in AI So what is Hell climbing now hill climbing is a heuristic search used for mathematical optimization problems it is used in the field of artificial science and basically this means that if you’re given a large set of inputs and a good heuristic function heuristic as in self-learning function it tries to find
A sufficiently good solution to the problem now this solution may not be the global optimal maximum or the best solution for the problem but it’s basically the best possible solution in a very reasonable period of time this implies that hill climbing solves the problems where we need to maximize or
Minimize a given real function by choosing values from the given inputs a very good example of this is the traveling salesman problem where you need to minimize the distance traveled by the salesman now the flowchart for hill climbing looks something like this first of all you select a current
Solution you evaluate that solution then you pick up a neighboring point or solution you evaluate the point that you just picked up then you make a decision is your new solution better than the original solution if yes you select the new solution as the current solution and
Then carry on the same method and if no if your original solution is better then again you go back to your step two you select a new solution from the neighborhood and then you evaluate X and this goes on and on now what you see on your screen right now is the algorithm
Which is corresponding to the flowchart that I just showed basically as you can see it is somewhat like a generate and test algorithm but it somewhat uses a greedy approach to reach a solution okay now you need to understand what the generate and test algorithm is
So as I mentioned hill climbing is a variant of the generate and test algorithm so first what it does is it generates possible solutions then it tests that solution or evaluates it to see if it is the expected solution if the solution has been found then it
Quits the loop else it goes back to step 1 and selects a new current solution it’s pretty simple isn’t it hence we can call hill climbing as a variant of generate and test algorithm as it takes feedback from the test procedure and then the feedback is utilized by the
Generator in deciding the next move in the search space another feature is that it uses the greedy approach so basically at any point in the state space the search moves in a Direction only which optimizes the cost of the function with the hope of finding an optimal solution
At the end so basically it’s going to take the best possible way to reach a solution now this is a State space diagram of hill climbing there are different regions in this state space diagram now for those of you who do not know what state space diagram is it is a
Graphical representation of a set of states that are search algorithm can reach versus the value of the objective function the objective function meaning the function which we wish to maximize or minimize now here the x-axis denotes the state space that is the state or configuration of our algorithm and
Y-axis denotes the values of objective function which is corresponding to a particular State the best solution will be the state space where the objective function has maximum value or the global maximum now there are different regions in this diagram first of all this is our current state it is the region in the
State space diagram where we are currently present during the search pretty self-explanatory isn’t it next you have the global Maxima now this is the best possible state in the state space diagram it is because that at this state the objective function has reached its maximum value next is the local
Maxima now this is a state which is better than its neighboring states however there exists a state which is better than this particular Maxima which is the global Maxima obviously we’ve just discussed it the state is better because here the value of the objective function is higher than its neighbors
But obviously we know it is not the best possible solution given reasonable time this is a good enough solution okay another kind of Maxima is a flat Maxima yes it is also known as a plateau now it is a flat region of State space where the neighboring states have the
Same value as you can see it is a straight line neighboring points are also on the same level then there is something you can see in the beginning it’s called a ridge it is a region Which is higher than the neighbor but it itself is a slope it’s a
Special kind of local maximum and apart from that that area in the front is a shoulder now if the plateau would not have descended from its sharp point and gone up it would have been called something known as a shoulder it’s basically an uphill Edge from the
Plateau yes with that we’ve come to the end of the introduction section let’s move on to types of hill climbing so first of all we have SIMPLE hill climbing now simple hill climbing is the simplest way to implement a hill climbing algorithm it only evaluates the neighbor node State at a time and
Selects the first one which optimizes the current cost and sets it as a current state it only checks its one successor State and if it finds that it is better than the current state then it moves to the next state else it will be in the same state it is less time
Consuming than the other types of hill climbing but it also gives a less optimal solution and the solution is not guaranteed now here is the algorithm for simple hill climbing first you evaluate the initial state if it is the goal State then you get success and stop
Step 2 you have to Loop until a solution is found or there is no new operator left to apply step 3 you select and apply an operator to the current state and finally you check the new state now if the new state is a goal State then
You return success and quit else if it is better than the current state then assign new state as the current state or else if it is not better than the current state then you return to step 2. and finally you exit that Loop as I said this might be less time
Consuming but it also gives the less optimal solution and the solution is not guaranteed next you have the steepest Ascent hill climbing now this algorithm is a variation of the simple hill climbing algorithm where it examines all the neighboring nodes of the current state and selects one neighbor node
Which is closest to the goal State now this algorithm consumes more time as it searches for multiple Neighbors so first you evaluate the initial state if it is the goal State then you return success and stop else you make the state which is currently where you are as the
Initial State pretty simple it is much like the previous type of Hell climbing next you Loop until a solution is found or the current state does not change now the conditions to this are many so let’s start with the first one let success be a state such that any successor of the
Current state will be better than it next for each operator that applies to the current state first apply the new operator and generate a new state evaluate the new state if it is the goal State then return it and quit else compared to success State now if this is
Better than the success State then set this as the new state of success if success is better than the current state then Set current state to success and then you can exit now what this does is it takes more time but since it is more complex you are most likely to get a
Better solution now apart from these two there’s a third kind of hill climbing it is the stochastic hill climbing now this algorithm does not examine for all its neighbors before moving rather the search algorithm selects one neighbor node at random and decides whether to choose it as a current state or examine
Another state that is it it does not go around searching the entire graph for a better node it just picks up points at random and decides to choose whether it is a better solution or not now a great way to optimize this particular type of hill climbing is to take as many
Possibilities in the bracket as possible it might take time but this guarantees you a better solution so as we discussed hill climbing is the most simple implementation of a genetic algorithm it completely gets rid of Concepts such as population and crossover instead it focuses on the ease of implementation it has faster
Iterations compared to most traditional genetic algorithms but in return it is less thorough obviously so even though it is not a challenging problem hello world is still a pretty good introduction so that is what we are going to do and we are going to execute it using python code
So how does it work hill climbing Works in a very simple way we can actually show it in a step-by-step list so you start out with an empty or a random solution this is your best solution make a copy of the solution and mutate it slightly now what you do is you evaluate
This new solution if it is better than the best solution we replace the best solution with this one if not you go to step 2 and repeat so basically to evolve a solution to a problem you need to write three functions you write a random solution you evaluate the solution and
Return a score and you mutate the solution in a random manner pretty easy isn’t it so for hello world let’s start with a basic outline of the hill climbing algorithm here you are trying to generate a random solution and you’re naming it the best solution next in your while loop you are trying
To print the best score so far while comparing your new solution to the previously best solution that there is next we are generating a random solution this function needs to return a random solution and in the hill climbing algorithm you make this a separate function now making this a separate
Function might be too much abstraction but if you want to change the structure of your code to be a population-based genetic algorithm it will be very helpful so here again we are giving a parameter of the length equal to 11 to generate the random solution and then we are returning the string
That we are getting next you evaluate the solution the target of our algorithm is producing the string hello world so our evaluation function is going to return a distance metric between two strings now this is a simple way to do it the function here will return the absolute difference of
Our solution to the Target and finally you mutate the solution now in genetic algorithms mutating a solution basically means randomly changing it in a small way in the context of this particular code this means that you change one of the letters randomly so that is what is happening in this
Piece of code now one last thing we need before our code is ready is the copy function but our solution is just a list of characters which is easily copied in Python so let’s get all of this code together tied all together in a state
That is ready to run now this is what the code looks like when you tie it all together I’m using a trusty Jupiter notebook for a complete jupyter notebook tutorial you can refer to the link given in the description bar so now let’s move
On to our code as you can see all the parts are there we’re starting out with generating a random solution then evaluating that particular solution mutating that solution to generate the best random solution and here is our base code now let’s bring it here and let’s try to run it
Here we are running all the cells one by one and once you run it you should be greeted with this particular output see how it is randomly changing the best score so far till you finally reach the solution that you were waiting for and that is hello world it’s right here
It starts out with these random solution it has taken 433 attempts to reach to our ideal solution and that my friends is how the hill climbing algorithm Works moving on let’s look at a few complexities and problems in different regions in hill climbing now hill climbing cannot reach the
Optimal best State Global maximum if it enters any of these following regions first up we have local maximum at a local maximum all neighboring states have values which is worse than the current state so since hill climbing uses a greedy approach it will not move to the worst state and it will
Definitely terminate itself at the local maximum the process will end even though a better solution may exist now to overcome such a problem you can utilize a backtracking technique maintain a list of visited States and if the search reaches an undesirable state it can backtrack to the previous configuration
And explore A New Path next it’s a plateau now on a plateau all the neighbors have the same value hence it is not possible to select the best Direction now to overcome this problem what you do is you make a big jump randomly select a state far away from
The current state and chances are that you will land at a non-plateau region next is a ridge now any point on a ridge can look like a peak because movement in all possible direction is downwards hence the algorithm stops where it reaches this state now to overcome this
Problem use two or more rules before testing it implies moving in several directions at once to test which one is the best Direction now let’s move on to a few applications of the hill climbing algorithm another hill climbing technique can be used to solve many problems where the
Current state allows for an accurate evaluation function such as the network flow traveling salesman problem 8 Queens problem and integrated circuit design hill climbing is used also in inductive learning methods it can be used in robotics for coordination amongst multiple robots in a team and these are only to name a few
The first artificial intelligence project is chatbot now chatbot is an AI software that can start a conversation or a chat with a user through messaging application websites mobile apps or even through calls chatbots are increasingly becoming popular many companies websites use chatbots to communicate with their customers it’s been used in almost all
The fields be it education medical I.T and even in banking websites now they’re using chat Bots for example Eva by HDFC bank now if you’re a beginner then you can program a simple version of a chatbot there are many chatbot available online just learn from them identify the
Basic structure and then build your own chatbot using that structure you can then enhance it using your creativity and make it better so this was the first AI project the next AI project idea is music recommendation app now due to AI music recommendation app which can also be known as music recommended engine
Makes it quicker and easier to show the music recommendation that a tailored to each user’s interests and preferences now how does this work so first it basically collects all the data which is what the songs the user listens to the most what is the Journal of the song
Which language that the user listens to and so on next it stores all this data and then analyzes it it then recommends songs from the similar genres and the same language and the song should have high ratings you would have seen this in apps like Spotify or Wing where they
Have the entire section of songs recommendation for you so using artificial intelligence online searching is improving as well since it make recommendation related to the user’s visual preferences rather than a project description you can program this music recommendation app by learning from some online blogs or watching some YouTube
Videos the next project idea is stock Tradition now many people invest in stocks and they need a stock predictor in order for them to know when to buy the stock NOW although it is impossible to predict the future we can make an estimation or guesses and an inform
Forecast based on the data we have in the present and the past regarding the stocks this is known as technical analysis which is used to predict the Stock’s price direction will it increase or decrease after a particular time so for your project you can create an application that analyzes the trend of
The stock market and offers data driven insights you can start off by keeping your stock prediction cycle small and then go on and try for higher values and insights also if you design a good shock prediction application there’ll be a great value and demand for such system
And it will make your career now moving on to a fourth AI project idea which is social media suggestions now artificial intelligence has been used in all popular social media networks that we use on a day-to-day basis like for example Facebook uses Ai and advanced machine learning to serve you all the
Content based on your preferences and to recognize people faces and photos so you can tag them basically and also Target users for the right advertisement also Instagram which is owned by Facebook uses artificial intelligence to identify visuals next LinkedIn uses artificial intelligence to offer job recommendation based on your qualification and interest
It also suggests people to connect with this also happens in Facebook so these were just some of the example of how social media uses artificial intelligence now ai powered research platform analyze a variety of social media analytics to understand which accounts can provide the most engagement reach and influence for a specific
Industry so for your project you can do any of the following tasks like suggested users to connect with people they might know or suggest them some content they might like to watch or suggest some product they might be interested in and so on so this was the social media suggestion project so now
Let us move on to next project idea which is to identify inappropriate language and hate speech now this project sounds easy but it is quite hard to identify all the hate speeches and inappropriate language there are many companies like Facebook Twitter and YouTube who are trying to create a
System like this so for your project you can use detection techniques which identify the characters in a context and then Compass it to the content that has already been removed as hate speech now usually that would be used for identifying any hate speech in any post like Facebook or Twitter post so design
An artificial intelligence system that looks into things like the text in a post the reaction coming to the post and how closely it matches the common phrases of a hate speech also if it contains at least one appropriate word then identify those words and report
Them so this could be one of your AI project now let us move on to the next AI project which is Lane line detection now many of you know that self-driving cars are gaining a lot of popularity now as a beginner it would be very hard to
Design this but you can design a part of it which is Lane line detection while driving this Leyland detection technique is used by many self-driving autonomous vehicles as well as line following robots so you can use computer visual techniques and AI to teach the vehicle
To go in a particular Lane you can use computer vision techniques such as color thresholding to detect the lane so usually the lanes are colored in white color and usually there are double Lanes in the middle of the road which operates the direction the vehicle runs in then
There is usually one white line at the end of the road after which is the edge of the road using all this data you can design an AI power system that detects the lean lines now let us move on to our next project idea which is monitoring crop Health artificial intelligence has
Been increasingly adopted as a past of agriculture industry Evolution using AI you can perform predictive analysis to determine what is the right date for sowing the seed to obtain maximum yield after the previous Harvest you can also get insights on the crop Health soil Health the fertilizer recommendation and
Also the next seven days weather forecast so you can create a project which uses artificial intelligence to monitor the health of the crop and check for Disease by using various images of the plant that has the same disease so when a user collects the image of a
Plant it will be matched with images that has already been stored and then diagnosed the particular disease and then even maybe provide a intelligent spraying technique and treatment automatically our next project idea is using AI for medical diagnosis AI has been used in medical industry for analyzing risk identifying hotspot and chronic diseases
And according for social determinants of health so for your project you can use artificial intelligence to develop a software that can be programmed to accurately spot signs of a certain diseases in medical images such as MRI scans or X-rays and CT scans for example you can design a system that uses
Artificial intelligence for cancer diagnosis by processing photos of skin lessons this project can be very helpful to diagnose patient more accurately and also prescribe the most suitable treatment the next project idea is AI powered search engine you can design a search engine which is powered by artificial intelligence which will scan
Billions of content available on the web and match the exact search sentence or keyword and will show the relevant information images videos texts and other documents you can also use ranking algorithms that will rank the content for a particular keyword based on various fact others like engagement rate
That is for how long that the user spends on that website is the content from a reliable website and so many factors to do this project you can refer some online blogs or watch some videos to get started also for this project you need to know a little bit about networks
And how the data passes on the internet from one place to another so this was about the AI powered search engine now let us move on to the next project idea which is AI powered cleaning robots today’s artificial intelligent powered robots possess no natural general intelligence but are
Capable of solving problems and thinking in a limited capacity you can design a robot that uses artificial intelligence to clean a room by scanning the room size identifying obstacles and remembering the most effective route for cleaning for starters you can design a robot that does only one of these things
Then you can enhance it until it effectively cleans the entire room properly the next AI project idea is how security now this is a very interesting project for this project you can design a system that uses artificial intelligence to scan and identify the face of the visitor first the facial structure of
The family members or someone who frequently visits the house can be scanned and stored so every time a visitor comes near the gate the system can scan the face and if it matches the existing facial structure that is stored in the database it can open the door and
Allow the person to pass else the gate can remain shut and the people living in their house can be notified that the person is waiting outside the next project idea is handwritten notes recognition handwriting node recognition refers to the computer ability to detect and interpret alphabets and numbers this
Inputs could be from various sources like paper document notes on the phone photos and other sources note that handwriting characters remain complex since different individuals have different handwriting trials so you can develop a system that uses artificial intelligence to scan the handwriting notes and convert them into digital
Format you can use the artificial neural network which is a field of study in artificial intelligence to design the system the next AI project idea is loan eligibility prediction nowadays one of the major problem banking employees face in this ever-changing economy is the increasing rate of loan defaults so the
Employees are finding it difficult to correctly Access Loan requests and decide whom to give loan and whom not to so in order to determine whether an individual should be given a loan or no you can create an AI program that will check a person’s loan eligibility criteria by accessing certain attributes
Of an individual such as the salary the previous loan details and so on and then make a decision to approve a loan or not this program will make the process lot more easier by selecting suitable people from a given list of candidate who have applied for loan so this was about loan
Eligibility prediction project idea so now let us move on to our next project idea which is AI powered voice assistant so this is one of the interesting artificial intelligence project idea you can create a voice based personal assistant using artificial intelligence so for this you have to train the system
To understand human language so it can understand and save the command in the database so next time you give the same command it will identify the words and perform the necessary action this can be very helpful and you can enhance it to do various activities like searching for
Some information or item on the web setting alarms taking down notes calling someone playing songs and many more the next AI project idea is e-commerce recommendation engine so in this project you can build an e-commerce recommendation engine using the similarities among the background information of the items or users to
Propose the recommendation to the user so in this project you can build an e-commerce recommendation engine using the similarities among the background information of the items or users to propose recommendation to the user so for example if the user has searched for Apple phones then you can design a recommendation engine that recommends
Only Apple phones to the user now the other way to do this is you can identify the trends and patterns in the previous and other user item interaction and advise similar recommendation to the present user based on its existing interactions so an example for this
Would be if a person has bought a formal shirt then you can design your recommendation engine to recommend more formal clothing and accessories you can use artificial intelligence to recommend the user what exactly they need the next AI project idea is AI enabled maps with artificial intelligence you can create a
Project that scans the road information and uses algorithm to determine the optimal route to take in order to reach the destination faster also to determine which mode of transportation is the best to go to a destination it could be on foot or in a car by bus or train you can
Also use Advanced artificial intelligence in the program by implementing Voice Assistant that will guide the users about Returns the potential roadblocks traffics and create augmented reality map into your time the next AI project idea is the motion detection now everything that’s happening in a science fiction movie
Could be your future there are varieties of field where artificial intelligence is used one such area of interest is detecting human emotions there are many top companies investing a lot of money in doing this so you can design a facial emotion detection and recognition system that can be used to identify human
Facial expressions so for this first the system would have to analyze the facial expression for some time and then perform facial feature extraction and classify the facial expression for starters you can design the system to identify only one expression maybe just happy or normal then you can enhance it
And try for different emotions the next AI project idea is AI Health engine you can create a project that will use artificial intelligence to give personalized Health guidance to a new user the user must provide all the medical reports and based on that the artificial intelligence system will check for any pre-existing condition
Ongoing health concerns and gaps in General Health knowledge then the health engine could be programmed to combine both the personal data of the users and the external help data to provide informal advice to the user it can also help the users with prescription support vaccination advice recommended doctor visits and specific condition guidance
So this was about AI Health engine project now let us move on to the next project idea which is trying on online clothes and accessories now you would have already seen this feature if you ever visited the lenskart app for your project you can design an artificial intelligence system that takes the input
Image and computes the person’s body model which would represent the posture and their shape the segments can then be selected on which the dresses are going to be displayed on like for example short on the body gloves forehands and so on and then when the user chooses a
Particular dress the system can combine them with a body model and update the image’s shape representation a 20th AI project idea is spam email detection spam email detection means detecting emails that are irrelevant to the user by understanding the text content of the email you can create a project that uses
Artificial neural network to detect and block spam emails also the newsletter or updates or any AD you can also enhance it let’s say the newsletters or updates or any ads which are received from emails can be liked by one person but disliked by another so you can include this feature using artificial
Intelligence that will filter the email based on the individual user preferences A lot of us are paranoid about how artificial intelligence might negatively impact our lives however the present picture is thankfully more positive I’ll be discussing how AI has impacted various Fields like marketing Finance gaming Agriculture and so on so let’s explore how artificial intelligence is helping our planet and at last benefiting humankind
So at number 10 we have artificial intelligence in artificial creativity now have you ever wondered what would happen if an artificially intelligent machine tried to create Music and Art here’s a short audio clip of a classical piece oh so this short audio was composed by an AI based system called Muse net now
Musenet is a deep neural network that can generate four-minute musical compositions with 10 different instruments and can combine styles from country to Mozart and to The Beatles musenet was not explicitly programmed with an understanding of music but instead it discovered patterns of Harmony Rhythm and Style by learning on
Its own another creative product of artificial intelligence is a Content automation tool called Wordsmith Wordsmith is a natural language generation platform that can transform your data into insightful narratives Tech Giants such as Yahoo Microsoft and Tableau are using word Smith to generate around 1.5 billion pieces of content
Every day let’s move on to our next field which is AI in social media now ever since social media has become our identity we’ve been generating an immeasurable amount of data through chats tweets posts and so on and whenever there’s an abundance of data Ai and machine learning are always involved
In social media platforms like Facebook artificial intelligence is used for face verification wherein machine learning and deep learning concepts are used to detect facial features and tag your friends deep learning is used to extract every minute detail from an image by using a bunch of neural networks machine
Learning algorithms are used to design your feed based on your interests another such example is Twitter’s AI which is being used to identify hate speech and terroristic language in tweets it makes use of machine learning deep learning and natural language processing to filter out offensive content according to a recent survey the
Company discovered and banned 300 000 terroristic linked accounts 95 percent of which were found by non-human artificially intelligent machines moving on to our next field we have ai in chat Bots now these days virtual assistants have become a very common technology almost every household has a virtual
Assistant that controls the home a few examples include Siri Cortana which are gaining popularity because of the user experience they provide Amazon’s Echo is an example of how AI can be used to translate human language into desirable actions this device uses speech recognition and natural language processing to perform a wide range of
Tasks on your command it can do more than just play your favorite songs it can be used to control the devices at your house book cabs for you make phone calls order your favorite food check the weather conditions and so on another example is a newly released Google’s virtual assistant called Google duplex
That has astonished millions of people not only can it respond to calls and book appointments for you it adds a human touch it uses natural language processing and machine learning algorithms to process human language and perform tasks such as manage your schedule control your smart home make
Reservations and so on next we have artificial intelligence in autonomous vehicles for the longest time self-driving cars have been a buzzword in the AI industry the development of autonomous vehicles will definitely revolutionarize the transportation system companies like waymo conducted several test drives in Phoenix before deploying their first AI based public
Ride healing service the artificial intelligence system collects data from the vehicle’s radar cameras GPS and cloud services to produce control signals that operate the vehicle Advanced deploying algorithms can accurately predict what objects in the vehicles vicinity are likely to do this may makes way more cars much more effective and safer another famous
Example of autonomous vehicles are Tesla’s self-driving cars AI implements computer vision image detection and deep learning to build cars that can automatically detect objects and drive around without human intervention Elon Musk the founder of Tesla talks a ton about how AI is implemented in Tesla’s self-driving cars and autopilot features
He quoted that Tesla will have fully self-driving cars ready by the end of the year and a robo taxi version one that can Ferry passengers without anyone behind the wheel Tesla’s autopilot software goes beyond driving the car where you tell it to go if you’re not in the mood for talking autopilot will
Check your calendar and drive you to your scheduled appointment that sounds pretty amazing moving on to our next application we have applications of artificial intelligence in space exploration so this is one of the most interesting fields in which artificial intelligence is being implemented space Expeditions and discoveries always require analyzing
Vast amounts of data artificial intelligence and machine learning is the best way to handle and process data of the scale so after rigorous research astronomers use artificial intelligence to go through years of data obtained by the Kepler telescope in order to identify a distant eighth planet solar
System this was accomplished by using AI technology artificial intelligence is also being used for NASA’s next Robo Mission to Mars which is the Mars 2020 Rover the Aegis which is an AI based Mars Rover is already on the red planet the Rover is responsible for autonomous targeting of cameras in order to perform
Investigations on Mars This proves how far AI has reached moving on to our next field artificial intelligence in the gaming field over the past few years artificial intelligence has become an integral part of the gaming industry in fact one of the biggest accomplishments of AI is in the gaming industry I’m sure
All of you have heard of deepminds AI based alphago software deepminds AI based alphago software which is known for defeating Lee saddle the world champion in the game of Go is considered to be one of the most significant accomplishments in the field of artificial intelligence shortly after the victory deepmind created an advanced
Version of alphago called the alphago zero which in turn defeated alphago in an AI to AI Face-Off unlike the original alphago which deepmind trained over time by using large quantities of human knowledge and supervision the advanced system alphago zero thought itself to master the game other examples of AI in
Gaming include the first encounter assault Rakin which is popularly known as fear is basically a first person shooter video game so what makes this game special the actions taken by the opponent AI are unpredictable because the game is designed in such a way that the opponents are trained throughout the
Game and never repeat the same mistakes so basically they get better as the game gets harder this makes the game very challenging and prompts the players to constantly switch strategies and never sit in the same position moving on to our next application we have artificial intelligence in banking and finance we
All know that trading mainly depends on the ability to predict the future accurately machines are great at this because they can crunch a huge amount of data in a short span machines can also learn to observe patterns and pass data and predict how these patterns might
Repeat in the future an example of this is Japan’s leading brokerage house namura Securities which has reluctantly been pursuing one goal that is to analyze the insights of experienced doctor Leaders with the help of computers so after years of research namura is set to introduce a new stock trading system
The new system stores a vast amount of price and trading data in its computer by tapping into this database of information it will make assessments for example it may determine that current market conditions are similar to the conditions two weeks ago and predict how share prices will be changing a few
Minutes down the line this will help to make better trading decisions based on the predicted market prices AI in banking is growing faster than you thought a lot of banks have already adopted artificial intelligence-based systems to provide customer support detect anomalies and credit card frauds an example of this is HDFC Bank HDFC
Bank has deployed an AI based chat bot called Eva which stands for electronic virtual assistant since its launch Eva has addressed over 3 million customer queries interacted with over half a million unique users and held over a million conversations Eva can collect Knowledge from thousands of sources and provide simple answers
And less than 0.4 seconds which is quite impressive moving on to our next field we have artificial intelligence in agriculture now here’s an alarming fact the world will need to produce 50 more food by 2050 because we’re literally eating up everything the only way this can be possible is if we use resources
More carefully with that being said artificial intelligence can help farmers get more from the land while using resources more sustainably Blue River technology has developed a robot called sea and spray which uses computer vision Technologies like object detection to Monitor and precisely spray we reside on cotton plants Precision spraying can
Help prevent herbicide resistance apart from this the berlin-based agriculture Tech startup called Pete has developed an application called plantix that identifies potential defects and nutrient deficiencies in soil by using images the image recognition app identifies possible through images captured by the user’s smartphone camera users are then provided with soil
Restoration techniques tips and other possible solutions the company claims that its software can achieve pattern detection with an estimated accuracy of up to 95 percent so the next field we’re going to talk about is artificial intelligence in healthcare when it comes to saving Our Lives a lot of organizations and medical care
Centers are relying on AI there are many examples of how AI in healthcare has helped patients all over the world IBM’s Watson for health is helping Healthcare organizations apply cognitive technology to unlock vast amounts of Health Data and power diagnosis IBM has also developed AI software specifically for medicine more than 230 Healthcare
Organizations worldwide use IBM Watson technology Google’s deepmind health is another such example that is working in partnership with clinics researchers and patients to solve real world Healthcare problems deepmind has successfully developed a system that can analyze retinal scans and spot symptoms of sight threatening eye diseases the technology combines machine learning and systems
Neuroscience to build powerful general purpose learning algorithms into neural networks that mimic the human behavior finally we have artificial intelligence in marketing we all know that marketing is a way to sugarcoat your product in order to attract more customers we humans are actually quite good at sugar
Coating but what if an algorithm or a bot is built solely for the purpose of marketing a brand or a company it would do a pretty awesome job for example let’s consider the recommendations provided by Amazon it’s a known fact that’s 35 percent of Amazon’s revenue is generated by its recommendation engine
Amazon makes use of AI and machine learning to recommend products to their customers it uses recommendations as a targeted marketing tool to increase their revenue there are different ways through which Amazon recommends products to you for example if you open up Amazon right now you’ll see a few sections like
These right you’ll see something known as your recently viewed items and featured recommendations here Amazon looks at the products that you’ve been browsing and recommends very similar products to you you’ll also see a section like customers who bought this item also bought this your Amazon studies the shopping
Behavior of customers who have a similar shopping Trend and displays items that are being purchased together in the past all of this is carried out by using artificial intelligence and machine learning algorithms another famous example of recommendation systems is Netflix Netflix uses machine learning to recommend movies to you based on the
Data it collects about you such as your browsing history your age your location and so on it is also a known fact that over 75 of what you watch is recommended by Netflix and what is the logic behind Netflix it is machine learning artificial intelligence and deep learning Artificial intelligence was coined in 1955 to introduce a new discipline of computer science it is rapidly and radically changing the various areas of our daily lives as the market for AI Technologies is demanding and flourishing now there is a significant race between many startups and internet Giants to acquire them
Now as we all know artificial intelligence is expanding and growing every day it’s literally taking over every sector or it’s spreading over every possible industry right now so let’s have a look at the top most trending Technologies of AI so on number 10 we have the robotic process
Automation or the RPA now robotic process automation refers to the functioning of corporate processes due to the mimicking human tasks and automates them now in this particular sphere it is important to bear in mind that AI is not meant to replace humans but to support and complement their
Skills and talent now companies like Pega systems automation anywhere blue prism uipath War Fusion Focus Etc work on these now RPA along with AI takes care of customer service accounting Financial Services Healthcare human resources Supply Chain management and a lot more but these are definitely some of the important aspects of the robotic
Process automation now let’s move on to the next one so on number nine we have text analytics and NLP now natural language processing or the NLP focuses on the interactions between human languages and computers it uses text analytics to analyze the structure of sentences as well as the interpretation
And intention through machine learning now this technology is widely adopted in fraud detection and for security systems many automated assistants and applications derive unstructured data by NLP now some of the service providers in this aspect include the basis technology expert system covio Indico live mind Breeze Etc now no wonder these terms
Make it to the top 10 trending artificial intelligence Technologies list so now let’s check out what’s on number eight here we have Biometrics now Biometrics is definitely a very common term to all of us right now because we use our fingerprints or the Biometrics and also face detection in order to
Unlock our phones laptops Etc now Biometrics deals with the recognition measurement and Analysis of the physical features of the body’s structure form and human behavior it Fosters organic interactions between machines and humans as it works with touch image speech and Body Language also it is predominantly used for the
Purpose of market research now we are effective agnesio phase first sensory Tazo all of these provide these technology service so this was about Biometrics now moving on on number seven we have cyber defense now this is definitely one of the trending Technologies AI because it is
One of the most important ones with the increasing number of cyber attacks so cyber defenses become very important in order to save our systems and also our confidential data now cyber defense is a computer defense mechanism that aims to detect prevent and mitigate attacks and threats to data and infrastructure of
Systems neural networks that are capable of processing sequences of inputs can be put to use along with machine learning techniques to create learning Technologies in order to reveal suspicious user activity and detect cyber threats so this was about cyber defense now on number six we have decision management now artificially intelligent machines
Have the capability of introducing logic to AI systems in order to gear them up to be used for training maintenance and tuning now in order to add value to the business and profitable decision management is already being used by organizations by incorporating it into their applications to propel and execute
Automated decisions some companies that provide this service are the Informatica Advanced systems Concepts Pega systems uipath Etc now next up on number five we have the marketing automation marketing has definitely become one of the most popular strategies for anything that you produce create or build right now because the right kind of marketing
Can make your product successful now marketing automation just makes this work simpler and better now the marketing and sales teams and divisions have adopted Ai and benefited a lot from it in return methods incorporating AI through automated customer segmentation customer data integration and campaign management are widely used the edexed AI
Has grown to become a Pioneer in adopting these marketing automation Technologies and in the coming days definitely most of the companies will only rely on the marketing Automation and that’s exactly what makes it one of the most trending Technologies in AI now next up on number four we have Digital
Trip this is one of the newest and very interesting concept of artificial intelligence now digital twins are just virtual replicas or physical devices that data scientists and it Pros can use to run simulations before actual devices are built and deployed they are also changing how Technologies such as iot Ai
And analytics are optimized now digital twin technology has moved Beyond manufacturing and into the merging walls of the iot artificial intelligence and data analytics as more complex Things become connected with the ability to produce data having a digital equivalent gives data scientists and other it professionals the ability to optimize deployments for
Peak efficiency and create other what-if scenarios now moving on to the next one on number three we have the industrial iot or the iiot now iiot which is the industrial indoor of things refers to the extension and use of The Internet of Things in industrial sectors and applications the iiot encompasses industrial applications
Including robotics medical devices and software defined production processes now both iot and iiot have the same main characteristic of availability intelligence and connected devices the only difference between those two is their General usages while iot is most commonly used for Consumer usage iiot is used for industrial purpose such as
Manufacturing supply chain Monitor and management system this technology is definitely for the advanced Industries and thus one of the most trending Technologies in AI right now now on number two we have virtual agents this is definitely very common to all of us because in our everyday life while
Booking a flight ticket or ordering a food anytime we need any help we are actually talking to Virtual agents online now a virtual agent is basically a computer generated animated artificial intelligence virtual character that serves as an online customer service representative it leads an intelligent conversation with users responds to
Their questions and also performs adequate non-verbal Behavior the introduction of virtual agents have been very helpful as it has reduced the work of human beings and also they provide assistance anytime and anywhere so this is about the virtual agents now moving on on number one we have augmented reality this is definitely one
Of the most fascinating Technologies in AI right now so augmented reality is actually an interactive experience of a real world environment where the objects that reside in the real world are enhanced by computer-generated perceptual information sometimes across multiple sensory modalities including visual auditory haptic and olfactory as well
Now basically augmented reality is a technology that superimposes a computer-generated image on a user’s view of the real world thus providing a composite view of what is happening now when we compare a VR and AR VR implies a complete immersion experience that shuts out the physical world that’s why it’s
Also called as the virtual reality but in case of AR that is augmented reality it adds digital elements to a live view Often by using the camera on a smartphone foreign at number one we have increased automation artificial intelligence can be used to automate anything ranging from tasks
That involve extreme labor to the process of recruitment that’s right there are n number of eibase applications that can be used to automate the hiring or the recruitment process such tools have to free the employees from tedious manual tasks and allow them to focus on complex tasks like strategizing and decision making an
Example of this is a conversational artificial intelligence recruiter called Mya this application focuses on automating tedious parts of the recruitment process such as scheduling screening and sourcing my is trained by using Advanced machine learning algorithms and it also uses natural language processing to pick up on details that come up in a conversation
It is also responsible for creating candidate profiles performing analytics and finally shortlisting applications it is a known fact that automating the recruitment process reduces time to high Higher by 50 percent and helps in finding key hires that impact profitability and growth our next benefit is increased productivity artificial intelligence has become a
Necessity in the business world it is being used to manage highly computational tasks that require maximum effort and time did you know that 64 percent of businesses depend on AI based applications for their increased productivity and growth an example of such an application is a legal robot I
Call it the Harvey Specter of the virtual world this bot uses machine learning techniques like deep learning and natural language processing to understand and analyze legal documents find and fix costly legal errors collaborate with experienced legal professionals clarify legal terms by implementing a AI based scoring system
On multiple scales it also allows you to compare your contract with those in the same industry in order to make sure that yours is a standard document moving on to our next benefit AI helps us in making smarter business decisions one of the most important goals of artificial
Intelligence is to help in making smarter business decisions Salesforce Einstein has managed to do that quite effectively following Albert Einstein’s dictum that the definition of genius is taking the complex and making it simple Salesforce Einstein is removing the complexity of artificial intelligence enabling any company to deliver smarter personalized and more
Predictive customer experience Salesforce Einstein is the comprehensive artificial intelligence for customer relationship management driven by Advanced machine learning deep learning natural language processing and predictive modeling Einstein is implemented in large-scale businesses for discovering useful insights focusing Market behaviors recommending the best possible solutions and also automating tasks
Moving on to our next benefit AI has been mainly used to solve complex problems that cannot be solved through other means throughout the years artificial intelligence has progressed from simple machine learning algorithms to advance machine learning Concepts such as deep learning this growth in AI has helped companies solve complex
Issues such as fraud detection medical diagnosis weather forecasting and so on consider the use case of how PayPal uses artificial intelligence for fraud detection thanks to deep learning PayPal is now able to identify possible fraudulent activities very precisely the company processed over 235 billion in payments from 4 billion transactions
By more than 170 million customers along with so much data machine learning and deep learning algorithms were used to mine data from the customers purchasing history in addition to reviewing patterns of likely fraud stored in databases the derived insights and patterns were then used to predict whether a particular transaction is
Fraudulent or not coming to the next benefit of artificial intelligence AI is used in strengthening the economy regardless of whether you think AI is a threat to the world it is estimated to contribute over 15 trillion dollars to the world’s economy by the year 2013. according to a recent report by PWC the
Progressive advances in artificial intelligence will increase the global GDP by up to 14 between now and 2030. it is also said that the most significant economic gains from AI will be in China and North America these two countries will account for almost 70 percent of the global economic impact the same
Report also reveals that the greatest impact of artificial intelligence will be in the field of healthcare and Robotics the report also precisely states that approximately 6.6 trillion of the expected GDP growth will come from productivity gains especially in the coming years major contributors to this growth include automation of
Routine tasks and development of intelligent boards and tools that can perform all human level tasks presently most of the tech Giants or already in the process of using AI as a solution to labor staffs however companies that are slow to adopt these AI based Solutions will find themselves at a serious
Competitive disadvantage moving on to our next benefit which is AI in performing repetitive does so we all know that performing repetitive tasks can become very monotonous and time consuming not to forget it’s quite boring so using artificial intelligence for tiresome and routine tasks can help us focus on the most important tasks in
Our to-do list an example of such an AI is a virtual Financial assistant used by the Bank of America called Erica Erica implements artificial intelligence and machine learning techniques to cater the Bank’s customer service requirements it does this by creating credit report updates facilitating bill payments and helping customers with simple
Transactions Erica’s capabilities have recently been expanded to help clients make smarter financial decisions by providing them with personalized insights as of 2019 Erica has surpassed 6 million users and has serviced over 35 million customer service requests our next benefit of artificial intelligence lies in personalization research from
McKenzie found that brands that excel at personalization deliver five to eight times the marketing Roi and boost their sales by more than 10 percent over companies that don’t personalize personalization can be an overwhelming and time-consuming task but it can be simplified with the help of artificial intelligence in fact it’s never been
Easier to Target customers with the right product an example of this is a UK based fashion company called thread that uses artificial intelligence to provide personalized clothing recommendations for each customer most customers would love a personal stylist especially one that comes at no charge but Staffing enough stylists for 650 000 customers
Would be expensive now instead the UK based fashion company thread uses artificial intelligence to provide personalized clothing recommendations for each of its system customers frequently take style quizzes to provide data about their personal style each big customers receive personalized recommendations that they can upward or download thread uses a machine learning
Algorithm called thimble that uses customer data to find patterns and understand the likes of the buyer it then suggests clothes based on the customer’s taste this is how personalization is performed in threat moving on to our next benefit which is artificial intelligence in global defense the most advanced robots in the
World are being built with global defense applications in mind this is no surprise since any Cutting Edge technology first gets implemented in military applications though most of these applications don’t see the light of day one example that we know of is the anbot the AI based robot developed by the
Chinese is an armed police robot designed by the country’s National Defense University capable of reaching maximum speed of 11 miles per hour the machine is intended to patrol areas and in the case of danger deploy an electrically charged riot control too the intelligent machine stands at a height of 1.6 meter and can
Spot individuals with criminal records the anbot has contributed in enhancing security by keeping a track of any suspicious activity happening around its vicinity moving on to the next benefit we have artificial intelligence in disaster management for most of us precise weather forecasting makes vacation planning easier but even the
Smallest advancement in predicting the weather majorly impacts the market accurate weather forecasting allows Farmers to make critical decisions about planting and harvesting it also allows Airlines to maximize the use of their planes it makes shipping easier and safer and most importantly it can be used to predict natural disasters that
Impact the lives of millions among companies using artificial intelligence to predict the weather only a few have invested as heavily as IBM after years of research IBM partnered with the weather company and acquired tons and tons of data the acquisition gave IBM access to the weather company’s impressive network of sensors and models
Providing a massive pipeline of weather data it could feed into IBM’s AI platform Watson in order to attempt to improve any Productions in 2016 the weather company claimed that their models used more than 100 terabytes of third-party data every single day the product with module is the AI based IBM
D Thunder this system provides highly customized information for business Lines by using hyper local forecast at a 0.2 to 1.2 miles resolution this information is useful for translation companies utility companies and even retailers moving on to the last benefit of artificial intelligence is that it enhances our lifestyle and we’re all
Aware of how AI is actually enhancing our life and changing our life in the last decade artificial intelligence has gone from a science fiction dream to a critical part of our everyday lives we use AI systems to interact with our phones and speakers through voice assistance like Siri Alexa and Google
Cars made by Tesla interpret and analyze their surroundings to intelligently drive themselves Amazon monitors are browsing habits and then serves up products it thinks we’d like to buy and even Google decides what results to give us based on our search activity artificially intelligent algorithms are here and they’ve already changed our
Lives For Better or For Worse but this is only the beginning and one day we look back at AI in 2019 and laugh about how primitive it was because in the future artificial intelligence is going to change everything thank you let’s look at some of the AI use cases
The first one is face detection and recognition using virtual filters on our face when taking pictures and using face ID for unlocking our phones or two applications of AI that are now a part of our daily lives the former incorporates face detection meaning any human face is identified the
Latter uses face recognition through which a specific face is recognized the next use case is text editor and autocorrect when you’re typing out documents that are inbuilt or downloadable auto correcting tools for editors that check for spelling mistakes grammars readability and plagiarism depending on their complexity
It must have taken you a while to learn your language before you became fluent in it similarly artificial intelligence algorithms also use machine learning deep learning and natural language processing to identify incorrect usage of language and suggest Corrections the next use case are chat Bots as a customer getting queries answered
Can be time consuming an artificial intelligent solution to do this is the use of algorithms to train machines to Carter to customers via chatbots this enables machines to answer frequently asked questions and track orders chatbots are taught to impersonate the conversational styles of customer Representatives through natural language processing Advanced chatbots no longer
Require specific format of input they answer complex questions requiring detailed responses the next use case is search and recommendation algorithms when you want to watch your favorite movies or listen to songs or perhaps shop online have you noticed that items suggested to you are perfectly aligned with your interests this is the beauty
Of artificial intelligence these smart recommendation systems learn your behavior and interest from your online activities and offer you a similar content the personalized experience is made possible by continuous training it is then able to predict your preferences by recommendations that keep you entertained without having to search any further
Moving ahead let’s discuss the AI roadmap for 2021 first let’s discuss based on Career a career in artificial intelligence is not one size that fits all so if you are interested to start a career in artificial intelligence here’s what you can do to start off get a bachelor’s
Degree in subjects like computer science Information Technology mathematics and statistics or even Finance or economics it’s the first step towards achieving your goal of having a career in artificial intelligence it is important to remember that to have a career in AI you need to have relevant skills along with your degree
Since artificial intelligence is the buzzword of today’s Tech world it is advisable to take up online and training programs to improve your skills ideally a bachelor’s degree could only help you land an entry-level job like Junior machine learning engineer or Junior data scientist so once you get a job and get into the
Industry and understand how things work you can plan your future career talking about positions and tailing supervision leadership or administrative roles you need to have a master’s degree or a PhD it’s best to have a master’s degree that offers Advanced Computer Science Education with a specialization in artificial intelligence or a master’s
Degree in artificial intelligence the master’s program generally focuses on developing professionals the robust coursework entails real world problems and application domain a relevant Masters or PhD degree can help you land a job at senior level like senior machine learning engineer or AI engineer a relevant master’s degree or PhD degree
Can help you land a job at a senior level like senior machine learning engineer or AI engineer or even robotics engineer with a good salary hike but even after landing a job it is important to fine-tune your technical skills to be an AI engineer one needs to
Be up to date with the latest skills and Technologies AI Engineers are not just skilled professionals but have in-depth practical and theoretical knowledge having a practical approach towards these technologies will help you gain an edge over other competitors additional add-on AI certification programs will win your brownie Point while seeking jobs in AI
Moving ahead let’s see the road map based on skills so let’s look at the skills you need to master the first skill required to become an AI engineer is programming to become well-versed in AI it’s crucial to learn programming languages such as python R Java and C
Plus plus to build and Implement models in-depth knowledge of computer software fundamentals starting from data structures trees graphs linear programming and computer architecture is required as the role of an AI engineer would be to simulate a machine to behave like a human hence without understanding the working principle of systems it
Would be difficult to cope the next skill is linear algebra calculus and statistics it is recommended to have a good understanding of the concepts of Matrix vectors and matrix multiplication moreover knowledge in derivatives and integers and their application is essential to even understand simple Concepts like gradient descent while statistical Concepts like mean
Standard deviation and gaussian distributions along with probability theory for algorithms like knife base and gaussian mixture are necessary to thrive in the industry the next skill is neural network architectures machine learning is used for complex tasks that are Beyond human capabilities to code neural networks have been understood and
Proven to be by far the most precise way of countering many problems like translation speech recognition and image classification playing a pivot role in AI Department the next skill is signal processing techniques competence and understanding signal processing and ability to solve several problems using signal processing techniques is crucial
For future extraction which is an important aspect of machine learning then we have time frequency analysis and advanced signal processing algorithms like wavelets curvelets and bandlets a profound theoretical and practical knowledge of these will help you solve complex situations the next skill is communication and problem solving skills AI Engineers need to communicate
Correctly to pitch their products and ideas to stakeholders they should also have excellent problem solving skills to resolve obstacles for decision making and drawing helpful business insights next let’s discuss the roadmap based on job roles so there are various job roles that you can get with your knowledge in artificial intelligence
So the first one is AI engineer artificial intelligence engineer are responsible for developing programming and training the complex networks of algorithms that make up AI so that they can function like a human brain this role requires combined expertise in software development programming data science and data engineering though this
Career is related to data engineering AI Engineers are rarely required to write the code that develops scalable data sharing instead artificial intelligence developers locate and pull data from variety of sources create develop and test machine learning models and then utilize application program interface calls or embedded code to build and Implement AI applications
The average salary of AI engineer per annum is 8 lakh seventy thousand rupees in India and one lakh 14 000 in the United States the next job role is machine learning engineer machine learning Engineers must possess strong software skills be able to apply predictive models and utilize natural language processing while
Working with massive data sets also machine learning Engineers are expected to know software development methodology agile practices and the complete range of modern software development tools right from Ides like Eclipse to components of continuous development pipeline the average salary of a machine learning engineer per annum is 11 lakhs rupees in
India and 1 lakh 14 000 in the United States the next job role is as a data scientist data scientists are big data Wranglers gathering and analyzing large set of unstructured unstructured data a data scientist’s role combines computer science statistics and Mathematics they analyze process and model data and then
Interpret the results to create actionable plans for companies and other organizations data scientists are analytical experts who utilize their skills in both technology and social science to find Trends and manage data they use industry knowledge contextual understanding and skepticism of existing assumptions to uncover solutions to business challenges
Average salary of a data scientist per annum is 10 lakh Rupees in India and one lakh 13 000 in the United States the next job role that we are going to discuss is Robotech engineer robotics can automate jobs but they require programmers working behind the scenes to
Ensure they function well as a robotics engineer their primary function is to build Mechanical Devices or robots that can perform tasks with commands from Human other necessary skills required for this role include writing and manipulating computer programs collaborating with other Specialists and developing prototypes the average salary
Of Robotics engineer per annum is 6 lakhs rupees in India and 84 000 in the United States foreign let’s get started with our basic level questions so first we have what is the difference between AI machine learning and deep learning I’m sure all of you have this question at the top of your
Mind because there’s a huge confusion between AI machine learning and deep learning so let’s try to understand how they are different now first of all AI came into existence at around 1950s alright this was followed by Machine learning and then deep learning was introduced now ai basically represents simulated intelligence in machines which
Means that it represents any robot or any machine that can mimic the behavior of a human being machine learning on the other hand is a practice of getting machines to make decisions without being explicitly programmed to do so now if you don’t program a machine how are you
Going to let it make decisions now the way machines learn is through data so the most important thing in machine learning is the data all right you’re going to train machines using data so that they can make their own decisions next we have deep learning now deep learning is basically the process of
Using artificial neural networks to solve complex problems so basically you can think of deep learning as a field that tries to mimic our brain okay so how we have neural networks in our brain that’s exactly how deep learning uses the concepts of artificial neural networks in order to solve problems now
Ai is a subset of data science so guys first of all data science is the process of deriving useful insights from data right it’s a process of extracting information from data that will help you solve problems so AI is a subset of data science now on the other hand machine
Learning is a subset of AI and data science because machine learning comes after AI so basically in AI you’re going to make use of techniques and concepts of machine learning in order to solve problems then we have deep learning so it’s sort of a hierarchy first we have
Data science then we have ai then we have machine learning and then we have deep learning deep learning is a subset of machine learning learning Ai and data science okay I hope this is clear now the main aim of artificial intelligence is to build machines in such a way that
They are capable of thinking like human beings all right so basically they must be able to mimic the behavior of a human being now the aim of machine learning on the other hand is to make machines learn by providing them a lot of data okay once you make a machine learn through
Data it’s going to be able to solve complex problems and find Solutions now the aim of deep learning is to build neural networks that are able to solve more advanced and complex problems okay now like I mentioned deep learning is like an artificial brain all right you’re basically building an artificial
Brain that is able to think exactly like how we do okay that’s what deep learning is it’s a little more advanced than machine learning now guys in short AI machine learning and deep learning are used to solve problems through data so basically AI makes use of techniques and
Method words of machine learning and deep learning to solve problems or to draw useful insights from data so this is the difference between AI machine learning and deep learning I hope all of you are clear with this now let’s look at our question number two the question is what is artificial intelligence give
An example of where AI is used on a daily basis so there are a lot of definitions of AI on the internet a few of them are artificial intelligence is an area of computer science that emphasizes on the creation of intelligent machines that work and react
Like humans so like I said basically a machine that is able to mimic the behavior of a human being is known as artificial intelligence another such definition is the capability of a machine to imitate the intelligent human behavior all right so artificial intelligence in short is basically a
Machine that we created who can act and think like a human being now where do you think AI is used on a daily basis there are tons of applications that make use of AI one of the most popular applications of AI is a Google search engine now if you just open up Google
Search and you start typing anything immediately you get recommendations these recommendations you derive by using machine learning algorithms by using deep neural networks and so on so on the top of my head the most General example of AI is the Google search engine all of us use Google search
Engine and we know how quick it is with its results and how relevant searches it gives us all this is because of AI all right now let’s look at our next question which states what are the different types of AI now a lot of people might not be aware of this
Because there are a couple of types of AI or a couple of types of machines which are hypothetical okay we haven’t actually implemented these machines in the real world we just have a theoretical definition of these okay let’s look at what I’m talking about so first of all we have reactive machines
AI now these machines are all based on the present actions okay they have no memory or they have no concept of storing memory so that they can learn from their experience they just react at the moment okay so they’re based on present actions and they cannot use previous experiences to form current
Decisions and update their memory then we have limited memory AI now this type of AI has some temporary storage of memory in it now if we have some memory stored in a machine we know that it can look back into the memory and it can try to make decisions based on previous or
Past experiences so limited memory AI makes use of that concept we have temporary memory here we do not have permanent memory but one of the top applications of limited memory AI is the self-driving cars I’m sure all of you have heard of Stack driving cars they
Make use of limited memory AI in order to run then we have theory of Mind AI now like I mentioned earlier there are a couple of types of artificial intelligent machines which are not actually implemented in the real world an example of that is theory of Mind AI
Okay this is basically an advanced machine which will have the ability to understand emotions people and other things in the real world we might have come close to this type of AI but we haven’t actually developed something that can understand emotions next we have self-aware ai now this is another
Such example of a machine that is not built in the real world this basically includes any machine that has Consciousness or that can react just like a human being okay so basically a machine that can take own decisions that can form on conclusions and these are machines that have the capability of
Making their own decisions without any human intervention now this kind of AI is not developed like I mentioned because it’s going to take up a lot of resources and we still haven’t reached that peak of evolution yet then we have artificial narrow intelligence now these are the general purpose AI that we see
On a daily basis I’m sure all of you have used Google assistance you’ve used Siri all of that comes under artificial narrow intelligence after that we have artificial general intelligence now these are a little more advanced than the artificial narrow intelligence then we have artificial superhuman intelligence now these are one of the
Most advanced type of AIS that are there now like I mentioned earlier there are a couple of types of artificial intelligent machines which are not actually implemented in the real world an example of that is artificial superhuman intelligence so Guys these were the different types of AI now let’s
Look at the next question which is explain the different domains of artificial intelligence now ai covers a lot of different domains starting with machine learning okay so machine learning like I mentioned earlier is the science of getting computers to act by feeding them data and by letting them
Learn a few tricks on their own without being programmed to do so okay so you’re not explicitly programming the machine instead you’re feeding it a lot of data so that it understands the data and it makes its own decisions then we have neural networks now neural networks are
Basically a set of algorithms or you can say a set of techniques which are modeled in accordance with the human brain okay like I mentioned earlier deep learning or neural networks is almost the same thing deep learning makes use of neural networks in order to solve complex problems now we have robotics
Now robotics is a subset of AI which includes different branches and applications of robots these robots are basically artificial agents which act in a real world environment okay so an AI robot works by manipulating the objects in its surrounding by perceiving moving and taking relevant actions then we have
Expert systems now an expert system is basically a computer system that mimics the decision-making ability of a human being now I know all of these domains sound very similar but they have a very different approach with which they solve a problem all right that’s the main difference between these domains next we
Have fuzzy logic systems now traditional systems usually give out output in the form of binary so usually if you feed something to a machine it’s always the binary form the output is also usually in the form of yes no true false and so on but when it comes to fuzzy logic it
Tries to give an output in the form of degrees of truth okay so it’s very different when compared to the traditional computer systems or the traditional programs next we have natural language processing now this is a field of AI that analyzes natural human language to derive useful insights
So that it can solve problems now NLP is used majorly in social media platforms so to better sentimental analysis is done via NLP even Facebook uses NLP in a lot of things all right so NLP fuzzy logic export systems machine learning neural networks and Robotics are the
Different domains of AI I hope all of you are clear with the domains now let’s look at our next question okay so how is machine learning related to artificial intelligence there is a huge confusion between machine learning and AI a lot of people tend to believe that Ai and
Machine learning is one and the same thing all right I would say that you cannot compare Ai and machine learning because machine learning is a subset of AI so basically AI makes use of machine learning algorithms and machine learning Concepts to solve problems that’s the basic difference or that is where the
Confusion ends machine learning is a technique which is implemented in artificial intelligence in order to solve problems I hope this is clear now let’s look at what are the different types of machine learning so there are three types of machine learning you have supervised unsupervised and reinforcement learning now supervised
Learning is the type of learning in which the machine learns by using label data now to make you understand let’s look at an example okay let’s say that you’ve input images of apples and oranges to your machine and you’ve labeled them you’ve told the machine
Like listen this is the Apple this is an orange and the output should also look like this okay so you’re labeling the input as apple and an orange and then you’re asking the machine to Output an apple and an orange but when it comes to unsupervised learning you’re not going
To label them you’re just going to give them images of apple and oranges and it has to figure out on its own it has to try and understand the difference between Apple and oranges try and understand how they look different or how they have a different color so basically in unsupervised learning you
Don’t have a label data set okay you’re going to give it an unlabeled data set and you’re going to ask it to find out and classify which is an apple and which is an orange okay that’s the difference between supervised and unsupervised now reinforcement learning is comparatively different let’s imagine that you were
Put off in an island okay let’s say that you were left in an isolated island what would you do now initially we’ll all panic and we won’t know what to do but after a point you’ll start exploring the island you start adapting to the change in the climate conditions you’ll start
Looking for food and then you’ll try and understand which food is right for you and which food is wrong for you you know you learn from your experience so in reinforcement learning basically an agent interacts with its environment by producing actions and discovers errors or rewards now the type of problems that
Supervised learning is used to solve is regression and classification when it comes to unsupervised it is Association and clustering and in reinforcement learning it’s all the reward based problems the type of data for supervised learning is labeled data for unsupervised it is unlabeled and for reinforcement it is no predefined data
Now when I say no predefined data I mean that the reinforcement learning agent has to start collecting the data so basically in reinforcement learning from data collection to model evaluation it does everything in terms of training supervised learning provides external Supervision in the form of label data set in unsupervised learning there’s no
Supervision that’s why it’s called unsupervised learning again in reinforcement learning there’s no supervision at all the agent has to figure everything out now how supervised Learning Works is you map the labeled input to the known output so basically you teach the machine like you tell it
That this is the input and this has to be the output when it comes to unsupervised learning you just provide data to the machine and it has to understand patterns and it has to discover the output now in reinforcement learning it has to follow the trial and error method third okay there’s no
Particular way in which the agent learns it just has to explore the environment try out a few things and learn from that experience popular supervised learning algorithms include linear regression logistic regression for unsupervised we have key means and for reinforcement learning we have Cube learning so Guys these were the different types of
Machine learning and I also discussed the difference between the three now let’s move on and look at our next question which is what is Q learning in the previous slide itself I told you that a type of reinforcement learning algorithm is Q learning so basically here what happens is an agent tries to
Learn the optimal policy from its past experience with the environment the past experience of an agent are a sequence of action State and rewards so what happens is first of all you take an agent and you put it in state zero okay let’s say there’s some state known as state zero
Now this agent is going to perform some action a naught on performing this action it is going going to get a reward R1 and if it gets a reward R1 then it’s going to move to State S1 but in case the action is wrong then it’s going to
Get a negative reward as in some points are going to be reduced so guys think of Q learning as a game you’re in state 0 and then you do some action and either you get a reward and go to the next state or else you lose and you go back
To the same state so until you learn you’re going to be in the same state but if you keep learning and if you keep receiving positive rewards then you’re going to move on to State one and similarly you move on to state two three and so on this is what Q learning is
About now the next question is what is deep learning now deep learning like I mentioned earlier basically mimics the way a brain works okay it learns from experience now the main concept behind deep learning is neural networks in our brain also we have neural networks so
What deep learning tries to do is it tries to use the concept of neural networks in order to solve complex problems so basically we’re trying to make our brain any deep neural network will have three types of layers the first is the input layer now this layer
Will basically receive all the input and it will forward them to the hidden layer now in the hidden layer all the analysis and the computation takes place all right once the computation is done the result is transferred to the output layer now there can be n number of
Hidden layers depending on the type of problem you’re trying to solve then we have the output layer so basically this layer is responsible for transferring the information from the neural network to the outside world so it’s as simple as that it’s pretty obvious input layer will take in the input hidden layer will
Perform the computations and the output layer will give out the output this is a small explanation of what deep learning is now of course this is much more complex than this but in short this is exactly what deep learning is now let’s look at our next question which is
Explain how deep learning works so basically deep learning is a concept based on something known as neuron okay neuron is a basic unit of the brain inspired from this neuron they came up with something known as perceptrons or artificial neurons now in this image on the left hand side you can see that
There is something known as dendrite these are modules which receive the input it basically receives all the signals that we send to our brain okay similar to the dendrites are the input layer in our artificial neural networks now in the previous slide we discussed that the input layer takes in all the
Input from the outside that’s exactly what a dendrite does so basically a perceptron receives multiple inputs it applies various Transformations and functions and then it provides an output so basically guys just like how our brain contains multiple connected neurons called neural networks we also have a network of artificial neurons
Called perceptrons to form a deep neural network so basically an artificial neuron or a perceptron it models a neuron which has a set of inputs Each of which is assigned some specific weight okay all of these inputs will have a specific weight and the neuron will compute some function on these weighted
Inputs and give you the output so the neuron will basically perform analysis and all of that on these weighted inputs to give you some output this is the basic concept of deep learning so there are inputs which have some weight on it and these inputs are then formulated and
Analyzed in order to give you an output now let’s look at our next question which is explain the commonly used artificial neural networks now this is a very theoretical question because in order to make you understand how each of them work will take a lot of time okay
So I’m just gonna briefly tell you what each of these networks are and what they do now feed forward neural network is the most basic kind of artificial neural network so basically the feed forward neural network is unidirectional the data passes through the input nodes and leaves through the output nodes in feed
Forward neural network usually the number of hidden layers depends on the complexity of the problem coming to convolutional neural networks here basically the input features are taken in small sets okay or they’re taken in batches this will help the network remember better because you’re feeding batches of images or you’re feeding
Batches of input to the neural network now this type of neural network is mainly used for signal and image processing next we have recurrent neural networks these are also known as long short-term memory networks so this basically works on the principle of feeding the output of a layer back into
The input layer in order to predict the outcomes okay this way is more precise and it is a little more complex when compared to convolutional Networks now one main important point of recurrent neural networks is that they have something known as memory so basically each neuron will have some information
Or some memory stored in them so that they can use this memory in order to take actions in the future so they have some experiences stored in the form of memory so that they can make that decisions based on previous actions now finally we have Auto encoders Now Auto
Encoders are mainly used in dimensionality reduction for learning generative models okay and one more important thing about Auto encoders is that the number of units in the output layer and the input layer is the same this is because the output layer has to reconstruct its own inputs so these were
The different types of artificial neural networks now let’s look at our next question which is what are Bayesian Networks okay so a Bayesian Network is a statistical model that represents a set of variables and the conditional dependencies in the form of a directed acyclic graph now basically on the
Occurrence of any event a Bayesian Network can be used to predict the likelihood that any one of several possible known causes was a contributing factor an example of this is a Bayesian Network could be used to study the relationship between diseases and symptoms so given a set of symptoms the
Bayesian Network can be used to find out the probability of the presence of any diseases all right so the next question is explain the assessment that is used to test the intelligence of a machine now guys this is a very common question and it is sort of a general knowledge
Based question right I’m hoping that most of you know the answer to this so let’s look at what the answer is I’m not sure how many of you have heard of Alan Turing so Alan Turing was the one who came up with the Turing test now this test is basically to determine whether
Or not a computer is capable of thinking like a human being so if a machine or if a computer passes this exam it means that that machine is capable of thinking like a human being it means that it is successfully an artificial intelligent machine meaning that it can make its own
Decisions and interpret data and form their own formulations or form their own conclusions about the data now sadly I don’t think there are a lot of machines that are past the Turing test in fact I’m not sure if there is any machine that’s positive during test as of now
But in the near future I’m sure that we’ll see machines who are more smarter than human beings and who have passed this test now for a machine it might be very easy to do computations but it might be very hard for a machine to just get up and walk around all right the
Simple things that us humans can do is very complicated for a machine they can do computations which we can do in probably a year they can do those competitions in maybe a week or less than a week but doing simple things such as walking up to the fridge or walking
Up to the kitchen is very hard for the machines so to achieve that level of intelligence we’re going to take a while but in the near future I’m sure BC machines which are way more capable than human beings now let’s move on to our next level now here I’ll basically be
Discussing intermediate level artificial intelligence questions so let’s look at the first question all right the first question is how does reinforcement learning work explain with an example okay so first of all a reinforcement learning is a type of machine learning we discuss about reinforcement learning earlier reinforcement learning is a type
Of machine learning wherein there’s an agent and you put this agent in an unknown environment all right now the agent has to figure out actions what sort of actions it must take and how it’s going to get rewards so that it can move from state to 0 to State one it’s
Sort of like a video game if you’re in a video game let’s say if you’re playing Counter Strike you’re in level 0 or state zero now if you perform some action and if you get some rewards you’re going to move to state one that’s exactly how reinforcement learning works
If you perform the relevant actions and the correct actions you’re going to get a reward and you’ll move on to the next state but in case you perform a wrong action you’ll get Negative rewards and you’ll stay in the same state unless and until you don’t learn all right so if
You learn and Achieve then you’ll move to the next state so basically a reinforcement learning system will have two main components it will have an agent and an environment now the agent have been repetitively saying agent an agent is basically the reinforcement learning algorithm it is the model the
Model has to learn everything on its own it has to collect data on its own it has to draw useful insights on its own okay you’re not going to feed any predefined data to this reinforcement learning agent all right he has to figure out everything on its own so let’s look at
An example of Counter Strike okay I’m not sure how many of you play the game but yeah what happens here is the reinforcement learning agent or the player one collects a state as not from the environment it okay so let’s suppose that you’re playing Counter Strike and
You’re in state zero now you’ll perform some action a naught right initially it’s going to be a random action so obviously if you’re put in an unknown environment your first action is going to be random correct because you don’t know what’s right you don’t know what’s
Wrong so in your state 0 you’ll take an action a naught this will result in a new state S1 and on achieving State S1 the agent will get a reward R1 okay from the environment now in the case of Counter-Strike games if you’ve observed whenever you win a state or you pass a
Level you’re gonna get some rewards maybe you’ll get more weapons or you’ll get more points okay just like that in reinforcement learning problem you’ll get some reward R1 okay it’s basically a plus point you might get a negative reward or a positive reward based on the
Action that you take now this Loop will go on until the agent is dead or it reaches the destination so in Counter Strike until you have failed the level you will keep playing the game right you’ll keep moving from State one state two state three and so on or if you’ve
Reached the destination exactly how it works in reinforcement learning if the agent has explored the entire environment and reached the end State that’s when the loop will end all right that’s exactly how reinforcement learning works it is very similar to the games that we play all right it’s very
Understandable now let’s move on and discuss the next question so the next question is explain markov’s decision process with an example now the solution for a reinforcement learning problem is achieved through the Marco’s decision process this is basically a mathematical approach that Maps the solution in reinforcement learning okay so now to
Understand this there are a couple of parameters in a Marco’s decision process they’re going to be a set of actions called a okay you can name them a a set of States there’s going to be reward there’s going to be policy and it’s going to be value to sum it up what
Exactly happens in a markov’s decision process is that the agent takes an action a to transition from the start state to the end State now while doing doing so the agent receives some reward R for each action that he takes the series of actions taken by the agent
Will Define a policy or an approach and the rewards collected will Define the value so the main goal in a markov’s decision process is to maximize the rewards by choosing the most Optimum policy meaning that you’re going to choose the best path or the best solution in order to get the most number
Of rewards now in order to make you all understand this better let’s solve the shortest path problem by using markov’s decision process I’m sure all of you have heard a shortest path problem this was I think taught to us when we were in 11th or 12th I’m not sure look at the
Diagram that is over here this is basically a representation of our problem given this representation our goal here is to find the shortest path between the node a and node D alright you can see nodes a b c and d we have to find the shortest path between node a
And node B now the link between these two nodes has a number on it okay for example between a and C you can see there’s a number 15. okay this basically denotes the cost to Traverse that edge so if you want to go from a to c you’ll
Spend around 15 points so our end goal here is to travel between node a and node D with minimal possible cost we should travel between a to d in such a way that our cost is minimal now in this problem if you notice that we have a set
Of States okay these are denoted by the nodes a b c d now like I mentioned earlier a Marco’s decision process has a set of States similarly in this problem the set of states are a b c Lee the action is to Traverse from one node to
The other so going from A to B is basically an action going from a to c is another action going from a to d is another action and so on now reward is represented by the cost on each of these links and the policy is the path which
Is taken to reach the destiny so our aim here is to choose a policy that gets us to node D in the minimum cost possible so how do you think you can solve this problem all right you can start off at node a and you can take baby steps to your destination now
Initially only the next possible node is visible to you like I mentioned earlier the initial action taken in a reinforcement learning problem is always random so at random you will choose any node let’s say you take a to B now if
You go from A to B you can go B to D and you’ll reach the destination so policy is the path which is taken to reach the destination all right so it can go from A to B to D or you can go from a to c to
D or you can go a c b d all right now it’s up to you to figure out which is the shortest path all right if you choose a path in such a way that the cost between a to d is minimized so guys this was a simple problem of how Marcus
Decision process is used to solve the shortest path problem now let’s move on and look at our next question all right now the next question is explain reward maximization in reinforcement learning so basically a reinforcement learning agent Works based on the theory of reward maximization okay in the previous
Question itself I told you that the main aim of reinforcement learning is to maximize the reward so that’s why a reinforcement learning agent must be trained in such a way that he takes the best action so that the reward is maximum okay this is exactly what reward
Maximization means he has to choose the best policy in such a way that the reward is maximum now let me explain this with a small game so in the figure you can see a fox you can see some meat and you can see a tiger now our reinforcement learning agent is the fox
His end goal is to eat the maximum amount of meat before being eaten by the tiger okay so he has to explore around eat the maximum number of meat that he can eat before the tiger kills him since the fox is a clever fellow he eats the
Meat that is closer to him okay so rather than eating the meat which is close to the Thai girl he eats the meat which is only close to him this is because the closer he gets to the tiger the higher are his chances of getting killed so as a result of this the
Rewards near the tiger even if they are bigger meat chunks will be discounted so because the fox is not going closer to the tiger and eating the meat chunks closer to the tiger this reward will get discounted now I know you all are wondering what this country is now this
Is done because of the uncertainty factor that the tiger might kill the fox so what is discounting of reward okay how does it work to understand this we Define a discount rate called gamma okay this is a parameter and the value of gamma always ranges between 0 and 1. so
The smaller the gamma the larger the discount and so on so guys this was reward maximization so here basically the fox will try to get as much as meat chunks as he can and he’ll also try to avoid getting killed because that will end the reinforcement learning Loop we
Also discussed the discounted Factor right now that is not needed to understand reward maximization but I just thought I’ll on some extra info now let’s look at the next question which is what is exploitation and exploration trade-off so basically exploration is like the name suggests it is about exploring and capturing more information
About an environment now on the other hand exploitation is about using the already known exploited information to heighten the rewards so consider the same example that we discussed in the previous question here the fox only eats the meat chunks which are close to him okay he does not eat the bigger chunks
Because even though the bigger chunks would give him more rewards it would get him killed okay he does not go towards the tiger itself now if the fox only focuses on the closest reward he will never reach the big chunks of meat okay this is what exploitation is he’s
Sticking only to the information that he knows and he’s trying to get the most number of rewards from it but if the fox decides to explore a bit it can find the bigger rewards okay the bigger awards are basically the big chunks of me and this is exactly what exploration is
Okay exploitation is about using the already known information to heighten your rewards exploration on the other hand is about exploring and capturing more information about an environment all right so that was about exploitation and exploration now let’s move on to our next question so this is a difference question which asks the difference
Between parametric and non-parametric models so a parametric models basically uses a fixed number of parameters to build the model now first of all guys what are parameters now parameters are basically predictor variables that are used to build a machine learning model or build any Predictive Analytics model
Now that we know what parameters are let’s try to understand the difference between a parametric and a non-parametric model now barometric model basically uses a fixed number of parameters to build a model a non-parametric model uses flexible number of parameters to build the model when it comes to a parametric model the
Assumptions about the data are very strong in a non-parametric model there are fewer assumptions about the data a parametric model has the fixed number of parameters everything is defined over here so the computation is very fast okay you know what sort of variables you’ll need to predict the outcome okay
You have a defined set of variables or a defined set of predictor variables that will compute your outcome so that’s why the computation is a bit faster when you compare to non-parametric models there are a lot of parameters taken into account now when it comes to a
Non-parametric model you do not have a fixed number of parameters all right you do not have a fixed number of predictor variables that will help you get to the outcome so the computation is a bit slower now parametric models require lesser data and non-parametric require more data example of parametric models
Include logistic retraction and naive bias and for non-parametric models we have KNN and decision tree models now logistic ni bias models are very Stern models because they have a fixed number of parameters or a fixed number of predictor variables and they will give you an immediate output okay when it
Comes to non parametric models like decision tree models and KNN you might even observe a little bit of overfitting okay this happens because you have a fewer number of assumptions about the data and also because your parameters are not fixed now that’s not the reason for overfitting but it’s seen that in
Some of the non-parametric models overfitting occurs more often now let’s discuss the next question which is what is the difference between hyper parameters and model parameters now model parameters are the predictor variables that I was speaking about earlier hyper parameters let’s discuss both Dr OKAY model parameters are the
Features of training data that will learn on its own during training whereas model hyper parameters are the parameters that determine the training process now let’s say that you want to determine the height of an individual depending on his weight the height and weight will become your model parameters
But your hyper parameter is basically the learning rate it’s the rate at which your model is going to learn this correlation between the height and the weight so this is the difference between model parameters and Hyper parameters model parameters are the ones that you find in your data these are all the
Variables that you use to predict your outcomes hyper parameters will Define your training process there is a huge difference between model and Hyper parameters now the difference is that they are internal to the model and their value can be estimated from the data hyper parameters are external to the
Model and their value cannot be estimated from data now like I said model parameters are derived from your data itself okay these are the parameters that are there in your data hyper parameters are the ones that you define in order to train your entire data so that is the difference between
Hyper parameters and model parameters so next question is what are hyper parameters in deep neural networks so guys like I mentioned in the previous example hyper parameters are variables such as a learning rate this will Define how your entire data training process goes for those of you who don’t know in
Order to build a model you first need to train the model and and then you need to test it okay now while training the model you’re going to make the model learn a lot of things you’re going to give it a lot of data it has to figure out relations between various variables
And how these variables are affecting the output all of this training will depend on a few variables such as the learning rate okay these are basically called hyper parameters in deep neural networks so these parameters will Define the number of hidden layers that are present in the network okay and more the
Number of hidden layers the more accurate your network is going to be whereas if you have lesser number of units you may cause under fitting in your data underfitting will also result in inaccurate predictions so that’s why you need to make sure that the number of hidden units in your hidden layers are
Perfect or ideal okay and this is determined by your learning rate or by your hyper parameters not by your learning rate specifically but by the number of hyper parameters you have these number of hidden layers are determined by the hyper parameters okay that’s why hyper parameters are very
Important in deep neural networks I hope you all are clear with this now let’s look at our next question which is explain the different algorithms used for hyper parameter optimization we’ll discuss the three methods which are grid search random search and Bayesian optimization okay now grid search
Basically will train the network only on the two sets of hyper parameters which are learning rate and the number of layers okay so it’s going to use every combination of these two sets in order to train the network okay after that it will evaluate the efficiency of the
Model by using the cross validation techniques cross validation is the best Improvement method okay it’s the best way to check if your model is optimal or not then we have random search now this will randomly select samples and it will evaluate sets for a particular probability distribution now in random
Search there is no fixed number of hyper parameters that is going to evaluate so will randomly select a set of hyper parameters okay for example now instead of checking your entire sample or your entire let’s say that you have 10 000 samples instead of checking all of these
Samples it will randomly select 100 parameters that can be checked okay and then it will use this to build the model after that we have Bayesian optimization okay now Bayesian optimization basically uses something known as the gaussian process basically the gaussian process will help in model tuning OKAY model
Tuning or you can also say parameter tuning so parameter tuning will help you tweak the parameters a little bit in order to improve the efficiency of the model okay so version optimization basically makes use of the gaussian process which will provide model tuning to your algorithm and thus improve the
Efficiency now guys the one of the most important ways to improve the efficiency of a model is by hyper parameter optimization okay if you’re tuning your hyper parameters and if you’re trying to check in which way these these hyper parameters will give you the most accurate outcome that’s when your result
Will be very good okay so that’s the best way to improve the efficiency of the model all right now let’s look at our next question the next question is how does data overfitting occur and how can it be fixed now guys this is a very common question in a machine learning or
In an artificial intelligence interview okay people expect you to understand what data overfitting is and how you can fix these problems okay because data overfitting occurs pretty often especially if you’re using decision trees or if you’re using random Forest okay random Forest actually reduces overfitting but sometimes with these
Complex models you can get data overfitting now to answer this question first of all let’s understand what overfitting really is so overfitting occurs when a machine learning algorithm captures the noise of the data okay this causes an algorithm to Show Low bias but High variance in the outcome now what
Overfitting really means is you have trained your model way too many times on the training data okay so basically the model has memorized the training data it has memorized the noise in the training data okay so if you feed new data to the model during the testing stage it will
Not be able to recognize the noise or it will not be able to recognize any sort of correlation in that data okay that’s why it won’t be able to get a proper outcome okay that’s when overfitting happens you have trained the model way too much with the training data and this
Has resulted in inaccurate outcome during the testing phase okay that’s what overfitting is about now how do you avoid overfitting first of all is cross validation now before this also I mentioned that cross validation is the best way to obtain a more optimal solution now the general idea behind
Cross validation is to split the training data in order to generate multiple mini train test splits okay these splits can be used to tune your model okay so you’re basically splitting the training data in such a way that you know the model does not just use the entire training data and memorize it
Instead it’s going to check the different sets in the training data and the different sets in the testing data and learn from it okay so cross validation is one of the best ways to prevent overfitting another method to prevent overfitting is by training the model with more data so feeding more
Data to the machine learning model will help in better analysis and classification however this method is not always going to work but yeah this is also one of the ways to prevent overfitting okay next we have removing features now many times the data set contains irrelevant features or
Predictor variables which are not needed for analysis such features will only increase the complexity of the model therefore it will lead to possibilities of data overfitting okay so if you have irrelevant data like for example if you’re trying to understand the weight of a person depending on its height and
You have another variable let’s say you have a variable like the name of the person okay now the name of the person is not relevant in understanding the height of an individual so if you have a relevant predictor variables then it will just increase the complexity of the
Model because you have an extra irrelevant variable all right this will only increase the complexity of the model it will not help the model in any way so make sure you remove irrelevant features or you remove redundant features okay the next method is early stopping now a machine learning model is
Trained iteratively this will allow us to check how well each iteration of the module performs but after a certain number of iterations the model’s performance starts to staturate further training will only result in overfitting okay so like I mentioned if you train the model with the same data and you
Make the model memorize the data then it’ll just saturate it it won’t be able to predict any outcomes after a point what you to do is you have to understand where you need to stop training the model so this can be achieved by using a mechanism known as early stopping so at
This point you know that you have to stop training the model because this might result in overfitting now regularization is one of the most common ways to prevent overfitting regularization can be done in N number of ways okay the method will always depend on the type of learner you’re
Implementing for example pruning is performed on decision trees now pruning is a type of regularization similarly the Dropout technique can be used on neural networks and also there are other methods like parameter tuning which can help to solve overfitting the next way to prevent overfitting is by using Ensemble modules now Ensemble learning
Is a technique that is used to create multiple machine learning models which are then combined to produce more accurate results so basically if you have one problem statement in machine learning you’re going to use like five to ten different models and then you’re going to calculate the accuracy
Depending on the average of the result from each of these models by this way you will reduce overfitting now in Sample models is one of the best ways to prevent overfitting an example is the random Forest random Forest uses Ensemble of decision trees to make more accurate predictions and to avoid
Overfitting so basically random force is a set of decision trees so here you’re going to train the model by using a set of decision trees and this way you will have different data sets and on each of these data sets you will have a different decision tree model okay this
Will reduce overfitting to a very large extent that’s why in most of the cases when you see a decision tree having overfitting issues you’ll be asked to use random Forest so guys those were the different ways to prevent overfitting now the next question is mention a technique that helps to avoid
Overfitting in a neural network now the most famous method to prevent overfitting in neural networks is Dropout technique okay now Dropout is a type of regularization technique which is used to avoid overfitting in a neural network so here what you do is you randomly select neurons and you drop
Them during the training phase right so the Dropout value also has to be chosen very carefully because a higher Dropout value will result in under learning by the network so if you’re dropping out too many predictor variables or if you’re dropping out too many neurons in
A neural network then the model will not learn enough okay because there’s not enough predictor variables and not enough neurons but if you have too much of a low rate for a dropout value then this might have a very minimal effect so make sure your Dropout value is very
Optimal depending on the problem you’re trying to solve okay so Dropout is the technique which is used to avoid overfitting in a neural network next question is what is the purpose of deep learning framework such as Keras transfer flow and Pie torch so Keras is basically an open source neural network
Library which is written in Python so basically it is designed to enable fast experimentation with deep neural networks now tensorflow is another open source software library for data flow programming tensorflow is mainly used in machine learning applications similarly pytorch is again an open source machine learning library for python its
Applications are mainly in the field of natural language processing now I’d say that these three deep learning Frameworks are the most important when it comes to machine learning and deep learning because they have a varied set of functions in them which help in building a better machine learning model
Or a better deep Learning Network now let’s look at question number 24 which is differentiate between NLP and text mining so guys NRP stands for natural language processing for those of you who don’t know now first of all let me clear out a confusion between text Mining and
Natural language processing a lot of people tend to think that text Mining and NLP are the same thing but text mining is the broader field and NLP is basically an application of text mining or it’s basically a technique used in text mining so the aim of text mining is
To extract useful insights from structured and unstructured texts whereas the aim of NLP is to understand what is conveyed in these texts now text mining can be done using text processing languages like Perl and NLP can be achieved using Advanced machine learning models such as deep neural networks now
The outcome for text mining is you’ll calculate the frequency of words you will understand the patterns between different words you will understand the correlations between two different words and you’ll see how these two words occur together more frequently and why they occur together more frequently so text
Mining basically will give you a more understanding about the words that are used in a document whereas in NLP you will understand the grammar behind the text you will understand in more depth about the language that is used in the document or in whatever you’re trying to analyze so that is the difference
Between NLP and X mining NLP is a little more advanced field because you use deep neural networks to perform this text mining on the other hand makes use of NLP next question is what are the different components of NLP now there are two components of natural language processing which is natural language
Understanding and natural language generation in natural language understanding you’ll basically map your input to some useful representation this means that you’ll try to understand the correlations in your language and it will also include analyzing different aspects of the language all right so this is majorly about understanding your
Text when it comes to natural language generation here you’ll understand how to generate text by having a brief plan about the text you’ll have sentence planning and you’ll have text realization now natural language generation will basically break down sentences or will break down text in order to understand it better okay
That’s what natural language generation is natural language understanding is more about analyzing your language or analyzing the text that you have at hand and predicting some useful outcome out of it generation is more focused on the planning aspect of your text so these are the different components of natural
Language processing now let’s look at what is stemming and limitization in natural language processing now what is stemming it is an algorithm which works by cutting off the end or the beginning of the word and only taking into account a list of com prefixes and suffixes that
Can be found in inflicted words now for example on the screen you can see that there is detections detected detection and detecting now if you apply stemming on these four words it will lead to detect okay because at the end of the day detection is detected detection and
Detecting is the same thing as detect so stemming will help you remove all of these unwanted prefixes and suffixes this way you can analyze the importance of the word right you don’t have to have extra suffix or prefix before the word now sometimes during stemming cutting of
The ends of the words will form an inaccurate result okay that’s why we have limitization in limitization the most important thing is the morphological analysis of the word okay so here in order to perform limitization you have to have a detailed dictionaries which the algorithm can look through and
It can form back to its Lemma so the main difference between stemming and limitization is that stemming will just crop the prefix and the suffix whereas limitization will try to understand the word in a grammatical way and give you an actual word as the output next is to
Explain the fuzzy logic architecture all right so the fuzzy logic architecture looks like what is shown on the screen okay so basically the input is fed into something known as the fuzzy fire okay the fuzzy Fire or the fuzification module will transform the system’s input into a number of fuzzy cells okay after
That it’s fed to the controller now the controller will have knowledge base and the inference engine knowledge base is basically a set of rules or you can say it’s an algorithm which is provided by experts inference engine light the name suggests will basically infer meaning out of these rules okay so once you’ve
Applied the rules to your input you’ll have to draw some useful insights or you’ll have to infer these inputs okay for that you use the inference engine after that whatever inferences an analysis you’ve formed from your inference engine is passed on to the defasification model now the defacification will just give you a
Crisp output all right it will give you a clear and cut output that is the whole fuzzy logic architecture now let’s understand the components of an expert system now there are three important components in an expert system which is knowledge base inference engine and user interface now like I mentioned in fuzzy
Logic the knowledge base and inference engine will play the same part the user interface is basically to provide interaction between the users of the export system and the export system okay the export system is basically a program that helps in decision making process okay so here the knowledge base will
Contain some high quality knowledge or contained rules and algorithms the inference engine will acquire all the knowledge that is needed to solve the problem and the user interface is just for the users to interact with the expert system okay this is the whole expert system component now obviously
This is a little more complex than this but let’s stick to how this works all right I’m just going to tell you the working of expert systems and fuzzy logic if I start to explain each and everything it’s going to take a lot of time all right so let’s move on to our
Next question which is how is computer vision and AI related now computer vision is a field of artificial intelligence that is used to obtain information from images or multi-dimensional data now computer vision is basically the concept behind the self-driving cars that you see these days right computer vision involves a
Lot of image processing so machine learning algorithm like gamings can be used in image segmentation support Vector machines can be used for image classification okay that’s how computer vision and AI are related because most of the things that happen in computer version like image processing and segmentation make use of machine
Learning algorithms like k-means and support Vector machines so to sum it up computer vision makes use of artificial intelligence Technologies to solve complex problems such as object detection image processing and so on that is the relationship between computer vision and AI now question number 30 is which is better for image
Classification is it supervised or unsupervised classification so guys earlier in the session we discussed what supervised learning is and what unsupervised learning is in supervised learning the images are interpreted manually by the machine learning expert to create feature classes now what this means is you’re manually going to feed a
Label set of data to the supervised learning model all right that’s how supervised Learning Works you’re manually going to feed a set of images which are labeled to the classifier in unsupervised learning the machine learning software creates feature classes based on image pixel values so basically in unsupervised classification
The model itself has to figure out what to do and what not to do okay so it’ll create our own feature class based on some values such as image pixels or it can also use the image color or it can use intensity factors in order to
Classify so if you ask me it is better to offer supervised classification because you’re manually inputting images with a lot more information okay whereas an unsupervised learning you’re totally letting the model perform everything okay so an image classification I think it’s better to go for supervised learning now let’s look at question
Number 31 the next question is finite difference filters in image processing are very susceptible to noise to cope up with this which method can you use so that there would be minimal distortions by noise now the noise is in an image can be due to high intensity or high
Contrast okay so if you increase the contrast and increase the intensity of an image you won’t be able to understand each pixel okay so each pixel will have a value Associated to it and if the intensity and the contrast of that pixel is a little too much it’ll be hard for
Us to understand the image properly it’ll be hard to perform image analysis because we don’t have a clear image contrast and intensity will just cause noise in an image so the best method to remove this is image smoothing okay it is used for reducing Noise by forcing
Pixels to be more like their neighbors okay this way you’ll have a favorite image or you’ll have a more equalized image now the next question is how is Game Theory and AI related so guys AI is actually applied in a vast number of fields okay so a lot of fields from
Computer vision to Game Theory to machine learning AI is always a concept behind these fields most of the game examples that we see make use of reinforcement learning or deep neural networks now deep neural networks and reinforcement learning are very closely related to AI because they are branches
Of machine learning so machine learning is majorly involved in Game Theory an example of this is in Dota 2 also they may choose a machine learning so game theory is just a very logical approach to solving a problem and machine learning is the best way to implement
Game Theory now question number three is what is the min max algorithm explain the terminologies involved in the problem now guys min max is one of the main algorithms which is used in Game Theory all right it is used to choose an optimal move for a player assuming that
The other player is also playing optimally meaning that both of these players are playing in order to win and you’re going to use the min max algorithm on one of these players so that they choose the optimal move in order to understand the min max algorithm you need to know what are the
Components in a game okay there’s something known as game tree it is basically a tree structure which contains all the possible moves in a game if it’s up down the right left any strategy everything is mentioned in the game tree now initial state is obviously the initial position of the player on
The board all right the successor function it defines all the possible moves that a player can make we’ll understand this in the next question itself so don’t worry if you haven’t understood this properly terminal state is obviously the end of the game it’s basically the state which will lead to
The end game or it’ll lead to your destination utility function is a numerical value for the output of the game so Guys these were the terminologies and this is what the Min Mass algorithm is it is basically a game theory algorithm which helps a player choose the best optimal policy in order
To win a game now I’ll explain this in more depth in the upcoming slides so let’s move on now the next couple of questions are going to be scenario based questions now such questions are very important in an interview because this is where the interviewer will understand
How well you know the concepts so the first question is show the working of the min max algorithm using the tic-tac-toe game now one of the major applications of the min max algorithm is the tic-tac-toe game okay you can understand and analyze all the possible outcomes of the tic-tac-toe game by
Using the min max algorithm let’s see how this happens now first of all in a min max algorithm or in a game there are two players involved okay the max is the player that tries to get the highest possible score and mini is a player that tries to get the lowest possible score
So this algorithm is designed in such a way that assuming that they’re going to be two players and obviously one player is going to win the game and that is a max player and Min is a player which loses the game and has the lowest possible score now the first step in the
Min max algorithm is to generate the entire game tree okay the game tree is all the possible outcomes that can happen in tic-tac-toe okay in the figure you can see that first X is a line in the first box then in the second box third and so on all the possible actions
That you can take in a tic-tac-toe game are put in this game tree and then step number two is to apply the utility function to get the utility values from all the terminal States getting utility value is important because this is how you’ll understand your outcome okay
You’ll understand if you’re going to win or lose now in the terminal States whatever numbers you see over here these are the utility values now step three is determine the utilities of the higher nodes with the help of utilities of the terminal nodes now in this diagram you
Can see that in the terminal nodes we have the utility values the step three is to get utility values in the higher stages which is the main stage all right these two circles you need to fill in the utility values by using the utility values which are in the terminal state
Now how do you calculate the utility value let’s start by calculating the utility value of the left node okay this red color node we’ll start by calculating this now you calculate that by finding the minimum of the three nodes that it’s leading to now this red
Node is leading to 3 5 and 10 and the minimum out of 3 5 10 is 3 so the utility value for this red node is going to be 3. okay similarly for this green node it’s going to be 2 because the minimum value between 2 and 2 is still
2. now step four is to fill in these utility values that you’ve calculated so now we have a min max algorithm which has all the utility values filled in now the only utility value which isn’t filled is the one with Max okay the one on the root node here we haven’t filled
The utility value again to fill this value you’re going to check the nodes which are directly connected to it which is three and two you’ll find the maximum between these two because this is the max function all right so here you’ll get a value of 3 so that’s why the best
Opening move for Max is the left node okay you can make use of the left node in order to win the game this is the first step that the max player has to take in order to get to the path of winning the game so guys by doing this
For each and every step you can win the game okay so you’ll have to calculate the utility value at the terminal nodes you’ll have to move up to the other hierarchical nodes above it calculate the utility values there until you reach the root node okay once you reach the
Root node you’ll get a utility value and that utility value will be connected to some move or some node you’ll have to take that node or you’ll have to take that move in the game in order to win the game so this way you’ll have to calculate the utility value for each and
Every move that the player makes so that the player will win the game so guys minimize algorithm is quite easy and is very understandable all you need to know is a little bit of math in order to solve this problem question number 35 is which method is is used for optimizing a
Min max based game now this is not a scenario based question but this question is usually asked if an interviewer asks you about a min max game now the best way to optimize a min max game is by using something known as Alpha Beta pruning now the main thing
About Alpha Beta pruning is that it will remove all the nodes that are not affecting the final decision it’s just a faster way to reach your outcome that’s what Alpha Beta pruning is all about so let’s look at an example to understand this okay let’s say there is another
Node over here okay here you can see that this is going down to a terminal state with utility value too okay now you don’t know the value of the other two nodes but if you use min max to calculate the utility of the other two
Nodes you’ll get a value of three so in this example again we’ll start at the terminal Norms so 3 5 10 are the utility values here so this will give us a value of 3 because we’re calculating the minimum over here now here you have 2
And you have two unknown values you have a or b okay I’ve named them as a and b okay let’s leave this for now let’s go to the next note okay here are the possibilities are 2 7 and 3 so the minimum between two seven three is two
Okay so here this is going to be three there’s going to be a value let’s say C and here this is going to be a value let’s say 2. now we know that the maximum between 3 C and something else will be three okay that’s because 2 is
The minimum value over here and the maximum will obviously be three so the hint here is in the 2 a b node we know that the value or the utility value will obviously be equal to 2 or it’ll be less than 2 because you’re calculating the minimum in this step now if you
Calculate the max out of these three values we’ll obviously get the answer as 3. so this way this entire node itself is remote because you don’t need it to get to the final answer okay that’s what Alpha Beta pruning is all about it will identify the nodes which are not going
To affect the fine final decisions and it will just remove those nodes so guys this is how the optimization for a min max game is done it’s done using the Alpha Beta pruning so that’s it for today guys I hope all of you enjoy the video have a great day
I hope you have enjoyed listening to this video please be kind enough to like it and you can comment any of your doubts and queries and we will reply them at the earliest do look out for more videos in our playlist And subscribe to any Eureka channel to learn more happy learning
-
Sale!
Wireless WIFI Repeater Extender Amplifier Booster 300Mbps
$29.99$14.99 Add to cartWireless WIFI Repeater Extender Amplifier Booster 300Mbps
Categories: Electronics, Wi-Fi Router, Wireless Wi-Fi Extender Tags: 300Mbps, 802.11N, Amplifier, Booster, Extender, mobile wi-fi booster, Remote, WIFI, Wireless, Wireless WIFI, Wireless WIFI Repeater, Wireless WIFI Repeater Extender, Wireless WIFI Repeater Extender Amplifier, Wireless WIFI Repeater Extender Amplifier Booster, Wireless WIFI Repeater Extender Amplifier Booster 300Mbps$29.99$14.99 -
Sale!
Full RGB Light Design Gaming Headset Headphones with Mic
$24.99$14.99 Add to cartFull RGB Light Design Gaming Headset Headphones with Mic
Categories: Electronics, Gaming, Gaming Headsets Tags: Design, Full, Full RGB Light Design Gaming Headset, Full RGB Light Design Gaming Headset Headphones, Full RGB Light Design Gaming Headset Headphones with Mic, Gamer, Gaming, Gaming Headset Headphones, gaming headset wireless, Headphone, Headphones, Headset, Light, Mic, Package, RGB$24.99$14.99 -
Sale!
Wireless BlueTooth Multi-Device Keyboard Mouse Combo
$39.99$19.99 Add to cartWireless BlueTooth Multi-Device Keyboard Mouse Combo
Categories: Electronics, Gaming, Gaming Keyboards, Keyboard Mouse Combos Tags: Combo, Keyboard, keyboard mouse combos, Mouse, MultiDevice, Set, WireKeyboard Mouse Combo, Wireless, Wireless BlueTooth Keyboard Mouse Combo, Wireless BlueTooth Keyboard Mouse Combos, Wireless BlueTooth Multi-Device Keyboard Mouse Combo, Wireless BlueTooth Multi-Device Keyboard Mouse Combos$39.99$19.99 -
Sale!
High Back Leather Executive Adjustable Swivel Gaming Chair with Headrest and Lumbar
$199.99$139.99 Add to cartHigh Back Leather Executive Adjustable Swivel Gaming Chair with Headrest and Lumbar
Categories: Gaming, Gaming Chairs Tags: Adjustable, Chair, computer chairs, Desk, Executive, Gaming, Girl, Headrest, High, High Back Leather Executive Adjustable Swivel Gaming Chair, High Back Leather Executive Adjustable Swivel Gaming Chair with Headrest, High Back Leather Executive Adjustable Swivel Gaming Chair with Headrest and Lumbar, High Back Leather Executive Adjustable Swivel Gaming Chairs, Leather, Lumbar, Office, Racing, Swivel$199.99$139.99 -
Sale!
Professional LED Light Wired Gaming Headphones with Noise Cancelling Microphone
$29.99$19.99 Select optionsProfessional LED Light Wired Gaming Headphones with Noise Cancelling Microphone
SKU: N/A Categories: Electronics, Gaming, Gaming Headsets Tags: Cancelling, Gaming, Gaming Headphones with Noise Cancelling Microphone, gaming headset, Headphones, Headset, LED, Light, Mic, Microphone, Noise, Professional, Professional LED Light Wired Gaming Headphones, Professional LED Light Wired Gaming Headphones with Noise Cancelling Microphone, Wired, Wired Gaming Headphones, Wired Gaming Headphones with Noise Cancelling Microphone$29.99$19.99 -
Sale!
Gaming Desk with LED Lights USB Power Outlets and Charging Ports
$349.99$249.99 Select optionsGaming Desk with LED Lights USB Power Outlets and Charging Ports
SKU: N/A Categories: Computer Desk, Gaming, Gaming Desk Tags: and Charging Ports, Charging, Desk, Desks, Gaming, gaming desk with led lights, Gaming Desks with LED Lights, Home, LED, Lights, Monitor, Office, Outlets, Port, Power, Room, Stand, USB, USB Power Outlets, White, Workstation$349.99$249.99 -
Sale!
Wired Mixed Backlit Anti-Ghosting Gaming Keyboard
$99.99$79.99 Add to cartWired Mixed Backlit Anti-Ghosting Gaming Keyboard
Categories: Electronics, Gaming, Gaming Keyboards Tags: Antighosting, Backlit, Blue, brown, Gaming, Gaming Keyboard, gaming keyboards, gaming keyboards and mouse, Keyboard, Laptop, Switch, Wired, Wired Mixed Backlit Anti-Ghosting Gaming Keyboard, Wired Mixed Backlit Anti-Ghosting Gaming Keyboards, Wired Mixed Backlit Gaming Keyboard$99.99$79.99 -
Sale!
Wireless Bluetooth 5.3 ANC Noise Cancellation Hi-Res Over the Ear Headphones Headset
$119.99$59.99 Add to cartWireless Bluetooth 5.3 ANC Noise Cancellation Hi-Res Over the Ear Headphones Headset
Categories: Electronics, Gaming, Gaming Headsets Tags: 5.3 ANC Noise Cancellation Hi-Res Over the Ear Headphones Headset, ANC, Audio, Bluetooth, Cancellation, Ear, Earphone, gaming headset, Headphones, Headset, Hi-Res Over the Ear Headphones Headset, HiRes, Noise, Wireless, Wireless Bluetooth 5.3 ANC Noise Cancellation Hi-Res Headphones, Wireless Bluetooth 5.3 ANC Noise Cancellation Hi-Res Over the Ear Headphones Headset, Wireless Bluetooth 5.3 ANC Noise Cancellation Hi-Res Over the Ear Headphones Headsets$119.99$59.99 -
Sale!
Wired Sports Gaming Headset Earbuds with Microphone
$19.99$9.99 Select optionsWired Sports Gaming Headset Earbuds with Microphone
SKU: N/A Categories: Gaming, Gaming Headsets Tags: Accessories, Earbud, Earphone, Earphones, Gaming, gaming headset with microphone, Headphones, Headset, IOS, Microphone, Sports, Wired, Wired Sports Gaming Headset Earbuds, Wired Sports Gaming Headset Earbuds with Microphone, Wired Sports Headset Earbuds$19.99$9.99 -
Sale!
150W Universal Multi USB Fast Charger 16 Port MAX Charging Station
$49.99$29.99 Add to cart150W Universal Multi USB Fast Charger 16 Port MAX Charging Station
Categories: Charging Stations, Electronics Tags: 150W, 150W Charging Station, 150W Universal Multi USB Charging Station, 150W Universal Multi USB Fast Charger 16 Port MAX Charging Station, 150W Universal Multi USB Fast Charger 16 Port MAX Charging Stations, 150W Universal Multi USB MAX Charging Station, 16 Port MAX Charging Station, 3.5A, Charger, Charging, Fast, laptop charging stations, Max, Multi, Port, Stand, Station, Universal, USB$49.99$29.99
Got a question on the topic? Please share it in the comment section below and our experts will answer it for you. For Edureka PG Diploma certification in AI & ML Curriculum, Visit our Website: https://bit.ly/3zDopRr Use code "YOUTUBE20" to get Flat 20% off on this training.
Thank u edurekha , it is very helpful 👍👍
Nice class, can I get a note or a book on this lecture.thank you
Thanks Edureka ! For excellence your team is😊 awesome
Thanks a lot, Edureka! Could you please provide the pdf notes for this course? Where can we send our mail id?
Love from Milky way.
Please provide pdf, souce codes to me
Great work. Can I get pdf of this please. Thank you
EXCELLENT AND AMAZING
It's helping me very much i am going to start with my btech degre in ai ml
I can say I am enjoying
Please send me the pdf of the presentation and data sets and the code
The course was soo good and understandable, can u please provide PDF files so it can be use to learn please
Good and useful sessions
where to get pdf?
Very informative and educational video…Thanks a lot team edureka…
Thank you Edureka amazing tutorial for beginners, Can you provide PDF notes or ppt🥰🥰🥰
Thank you team edureka , your content is amazing and well explained.
Input 🔣🔠
Amazing tutorial for beginners. Thank you so much, I appreciate it a lot.
Hey edureka thank you for providing the excellent content and plz provide the PPT for this
Hey Edureka thankyou for this wonderful session, can you please send the PDF of this presentation ?
Thanks team edureka for such type of content for free, can you provide a PDF notes or ppt for AI.
Thank you for this wonderful video on AI. Can you provide me the ppt?
Hi thank you for this can I request for pdf presentation
thanks!
can we get the pdf of the presentation
?
thank you for providing the wonderful tutorial. How long can the master program last?
Could you please make a detailed 10 hour course on computer networking?
Thank u so much for sharing such an amazing content❣❣