Build A Python App That Tracks Amazon Prices!
- March 4, 2024
- Posted by: MainInstructor
- Category: Go JavaScript Python VSCode
Video Title: Build A Python App That Tracks Amazon Prices!
Okay if you’re poor you’re gonna love this episode so ever since I did this YouTube thing I really wanted to get a new camera and that’s the Sony a7 marquetry not that that matters but it costs so much money two thousand euros so the plan was I’m gonna build a small
App that checks the price on Amazon and if that price falls down whether whatever I want basically like $1,600 then it’s gonna automatically send me an email that’s what we’re gonna build we’re gonna flex our snake skills okay our Python skills and then we can celebrate and we still don’t have enough
Money together to buy the damn product so yeah you can use this all multiple websites whatever you prefer let’s stop talking and let’s get going okay let’s get going so I just have a scrape or dot pie opened up here it’s just an empty empty Python script okay
So you can do this on your own and vs code and what we need to do is actually install two separate things so we can open up here oh my god this is from the finished project okay so what you can do is say NPM actually NPM is it’s not what
We want okay we’re working with Python we can say pip install requests and ps4 all right I’m gonna explain what these two things do so install those two after you do that we can import let’s say import requests and what this basically does is we can get a page we can access
A URL so maybe Amazon and we can pull out the actual data from that website okay so all I’m gonna do is I’m gonna say URL I’m gonna set this equal to and the one I used is this from amazon.de so as you can see here I just saw a Sony a7
You can get this one to amazon.de search for Sony a7 I think it’s the first or second link if you want to follow along with exactly this yeah and we can paste this back in here and our URL perfect now I’m not every website is gonna work
With web scraping some block it some don’t this one works so this is the one we’re gonna use okay that’s the first thing the second thing I want is headers so I’m gonna set this equal to a dictionary here I’m gonna say user agent alright and this is just
Basically gives us some information about our browser and we need to pass this in to our header so how can you get it you can search for my user agent all right and you can just copy this and paste it in so let me just paste this in
Here like so perfect that’s the second thing I have and now we can actually make a call so I can say page equals two requests all right the library that we’re using yet and I can get the URL and the second argument is gonna be requests equal not
Requests my bad headers equal to the headers okay that’s it that’s all we need perfect and this actually returns all the data from that website now what beautifulsoup does for us is we can parse it and we can pull out individual items from it so I can say from PS for import beautifulsoup
Okay now if you’re from JavaScript you will do this reverse so it would be imported off from well in Python we are cooler and we do the opposite way okay so once we have that I can create another variable called soup I can set this equal to beautiful soup and passing
The page thought content and the second argument is gonna be a string called HTML dot parser alright this is going to parse everything for us and now we can actually pull out individual pieces of information if you want to see what this did actually we can I can show you we
Can say print soup and there’s a method on it called prettify all right let’s run this we can call scraper dot python scraper dot pi ok run this take a look prove all my goodness hacker man there’s so much it’s not stopping it will never stop just like my
Mental state ok so there we go take a look we have lives here with spans and everything I don’t know Amazon’s crazy and we’re just gonna close this up okay so what we can do is we can go back here press f12 and we can just pull out any data we want I
Can hit this icon here click on this take a look we have a spam with a product title how can I pull that information out I can create a div here called not a div a variable called title I can just say soup thought find all
Right and you have a lot of different ones that you can experiment with as you can see you can have name attributes and you can have a class in here if you want search by class and we can also do by ID that’s what I’m gonna show you so I can
Say find IB and set this equal to product title because that’s how it’s named there now this actually if we print this out title this actually returns me behold div I believe yeah take a look we have a span here and we have D span with the actual information
And there’s a lot of empty white spaces for some reason Amazon what she’s doing I’m not sure okay so to actually get the text out of it we can do dot get text here alright that’s only gonna return the text but we still have that problem of
Hemming having a lot of empty spaces here as you can see so we can actually strip that out as well and how do you do that well my saying dot strip I just said it so there we go let’s do another Python scrapper dot pie take a look and we should get back
Only Sony Alpha blah blah blah perfect now I don’t actually need this but hey why not let’s have it here what I actually want to have is the price so let’s analyze the price click on the price it has an ID of price block our price so that’s exactly what I’m gonna
Do I’m gonna say price is equal to soup dot find I’m gonna say ID equal to price block our price all right make sure you don’t misspell this and we can also extract the text here and get text now the problem here is that this is actually a string alright and we cannot
Really compare the values I cannot compare a number to a string so we need to actually modify this in just a bit so how can we do that well first of all what I can do is I can add a converted price instead that equals you and I can just extract
The first five characters are I so I can say price add these break brackets and say zero to five and all this does is it checks the first five so it’s gonna be one two three four five so it only extracts this information it doesn’t extract the dots ninety nine with the
Euros as well all right so if we console.log this out friends converted price let’s take a look at what we have run this and it’s gonna give us back one point nine eight nine all right otherwise it would give us back 1.98 9.99 and a euro symbol okay so that’s
Why we did this so we only have back the first five characters but it’s still a string which is not good so what we can do is we can add a float here wrap this round in the float and now it turns it into a float if you don’t know what a
Float is well basically an integer and int is a whole number so like five and a float is like five point eight nine all right so you can add a dot to it okay so we have this back or converted price so what we actually want to do
Here is something like this if our converted price is smaller than a thousand eight hundred or whatever you want the price to be like that then I want to send an email all right so I’m gonna say send mail perfect so that’s what I want to happen okay and we can
Actually leave everything like so what I’m gonna do is take everything we have up here and wrap it inside a function so I’m gonna say def define check price all right this is just a function and I’m gonna add everything that we have from here inside this function alright get
Rid of some empty spaces perfect so we need to define the send email so I’m gonna just go down here say def send email it’s gonna be another function and we’re gonna take a look at how we can actually send an email with the updated
Price so the first thing we need to do we’re gonna do this with Gmail and for this to work we’re gonna have to enable the 2-step verification there’s another way we can do it the simpler way let me quickly show you some what we can do I’m
Gonna pull this up here is if you search for something like Google allow less secure apps or something like that let’s scroll down here let’s secure apps Google account ok so you can enable this one there’s gonna be in the enable button and that’s gonna work just fine
You can use your gmail password and you can send emails just fine if you don’t want to use your gmail password what you can do is you can enable these 2-step verification so just Google 2-step verification or whatever it is called so here or the second link alright just
Enable here 2-step verification and once you do that what you can do is you can google for Google app password alright and you can create passwords for different things I’m gonna sign into my Google account and as you can see you can create one you can hit here select
App and you can create a generate a new password for your email for you your YouTube calendar whatever you want so in this case you would select mail and you would select Windows computer all right that’s what I’m on you might probably have Mac or whatever I’m gonna delete
This old one I had and I’m gonna generate alright so this just generates us a new password that we can use whoo okay let’s go back to our scraper and see what’s up ok so we actually need to import another package here we don’t need to install
This separately so what we can do is we can just go up here and say import smtp lib alright SMT flip what this is is basically a simple mail protocol or something like I’m not sure exactly what it’s called but it enables you to send emails now let’s go down here and what
We need to do is actually establish a connection between our connection and Gmail’s connection so to do that we can say set up a server here and set it equal to SMT be blip dot SMTP and what we need to pass here is smtp.gmail.com all right so
This is Google’s or Gmail’s SMTP and the connection number is gonna be eight five eight seven all right that’s what we need to pass in and then what we’re gonna do is we’re gonna say server dot hello all right if you don’t know what in lo
Is let me pull it up for you here on the screen basically it’s a command sent by an email server to identify itself when connecting to another email all right so it’s kind of establishes a connection between D two again so we need to call
That after we do that what we need to do is go down here and say server dot start TLS which basically encrypts our connection and then we can call server dot hello again all right I know this is getting pretty pretty jargony but hey that’s what we
Need to do so that’s what we’re gonna do if you don’t remember this you can just always come here and check so next up what we want to do is we actually need me to login so we can call server down login the first argument is gonna be the
User as you can see and the second one is gonna be the password so my user is gonna be ad magician I know that gmail.com and the second one is gonna be the password now again it depends what you did if you did the 2-step identification then you can just
Generate this password which is exactly what I’m gonna do okay so I’m gonna pass this in here we have the connection and now we can actually set up our email I can add a subject set this equal to hey the price fell down and the second one
Is gonna be a body set that equal to check the Amazon link right and I actually can copy this link down here so our email actually gives us dealing back with the product perfect and then we can set up a message I’m gonna format this string by adding an F
Which means we can just basically interpolate subject here and just add subject all right kind of like JavaScript with the dollar sign and curly braces and then we can add a new line new line and say body okay which is the actual message perfect and then we
Can send the email we can say server dots and mail and here what we need to pass in as from as you can see from – and the actual message so from I’m gonna say add magician and gmail.com and the second one is gonna be I’m gonna send it
To myself again so – another email I’m gonna say my full name here hi mouth and these third argument is gonna be the actual message which we did up here so from – and the message and finally I’m just gonna print out that hey email has been sent
Yeah perfect and finally we need to close up D connections I’m gonna call server da quit quit my life okay so that’s what we have so now up here what I can say is hey if the converted price falls below at one point seven thousand all right so this is like a thousand
Seven hundred euros oh the JavaScript then we can send an email that’s it that’s how we need to do now in this case this is not gonna work because our price is this one is bigger so let’s give it a shot boom well what we actually need to do is
Call this function this check price because we edit it into a function so we’re not actually calling anything here as you can see so let’s go down here and just call this function okay let’s take a look huh please all right so we get back the price and the title because we just
Printed that out on the screen but nothing really happens we don’t get this Hey email has been sent however take a look if our price changes in the future so I’m gonna just add a bigger sign here to make this true all right because this
Is bigger but you can add any price you want here well now this is gonna be true and it’s gonna send me an email so let’s take a look boom boom boom come on hey email has been sent whoa let’s take a look here let me go to my
Email and prove prove it to you so I’m gonna zoom in here oh never mind all right hey so take a look boom boom boom email has been sent price has fell down and we get the Amazon link to it so everything works just fine
Perfect now what we can do is we can actually make this run for longer so multiple times and to do that what we can do is we can just import something this is again you don’t need to install this we can just import time and let’s
See what we do rather than just running this once I can just add a while loop here while true so this is always gonna run like that what I can do is say check price and then I can add a time sleep and I can add the number of seconds I
Want this to sleep so basically it runs once and then it pauses the execution for a number of seconds so if I add 60 here then it’s gonna pause it as you can see delay the execution for a given number of seconds so this would check every minute for you again however
That’s probably not a good idea so you would want to check this at least like maybe once a day so you would do I don’t know my math is really bad so you would do I mean 60 times 60 times 60 would be probably an hour so there you go now you
Acquired the skill of scraping the web and also sending some simple email so I highly recommend you to try it out with different websites see what information you can get now again not every website is gonna let you do this some have some checks anti web scraping
Kind of checks with future CAPTCHAs or whatever so not everything is gonna work out unfortunately but yeah experiment and also don’t send like a thousand requests to mess around with your server yeah have fun with it I also did one with getting all the recent movies of
The IMDB so the latest movie releases and it also saves it down into a file for you a CVS file if you want to see that I might put it up on patreon just leave a comment in the description and I might upload these source files for that as well
Ok thanks again so much for watching I apologize for no magic tricks for today I’ll do this one because just super simple and stupid but I’ll catch you in the next one have a great day bye
-
Sale!
Wireless WIFI Repeater Extender Amplifier Booster 300Mbps
$29.99$14.99 Add to cartWireless WIFI Repeater Extender Amplifier Booster 300Mbps
Categories: Electronics, Wi-Fi Router, Wireless Wi-Fi Extender Tags: 300Mbps, 802.11N, Amplifier, Booster, Extender, mobile wi-fi booster, Remote, WIFI, Wireless, Wireless WIFI, Wireless WIFI Repeater, Wireless WIFI Repeater Extender, Wireless WIFI Repeater Extender Amplifier, Wireless WIFI Repeater Extender Amplifier Booster, Wireless WIFI Repeater Extender Amplifier Booster 300Mbps$29.99$14.99 -
Sale!
Full RGB Light Design Gaming Headset Headphones with Mic
$24.99$14.99 Add to cartFull RGB Light Design Gaming Headset Headphones with Mic
Categories: Electronics, Gaming, Gaming Headsets Tags: Design, Full, Full RGB Light Design Gaming Headset, Full RGB Light Design Gaming Headset Headphones, Full RGB Light Design Gaming Headset Headphones with Mic, Gamer, Gaming, Gaming Headset Headphones, gaming headset wireless, Headphone, Headphones, Headset, Light, Mic, Package, RGB$24.99$14.99 -
Sale!
Wireless BlueTooth Multi-Device Keyboard Mouse Combo
$39.99$19.99 Add to cartWireless BlueTooth Multi-Device Keyboard Mouse Combo
Categories: Electronics, Gaming, Gaming Keyboards, Keyboard Mouse Combos Tags: Combo, Keyboard, keyboard mouse combos, Mouse, MultiDevice, Set, WireKeyboard Mouse Combo, Wireless, Wireless BlueTooth Keyboard Mouse Combo, Wireless BlueTooth Keyboard Mouse Combos, Wireless BlueTooth Multi-Device Keyboard Mouse Combo, Wireless BlueTooth Multi-Device Keyboard Mouse Combos$39.99$19.99 -
Sale!
High Back Leather Executive Adjustable Swivel Gaming Chair with Headrest and Lumbar
$199.99$139.99 Add to cartHigh Back Leather Executive Adjustable Swivel Gaming Chair with Headrest and Lumbar
Categories: Gaming, Gaming Chairs Tags: Adjustable, Chair, computer chairs, Desk, Executive, Gaming, Girl, Headrest, High, High Back Leather Executive Adjustable Swivel Gaming Chair, High Back Leather Executive Adjustable Swivel Gaming Chair with Headrest, High Back Leather Executive Adjustable Swivel Gaming Chair with Headrest and Lumbar, High Back Leather Executive Adjustable Swivel Gaming Chairs, Leather, Lumbar, Office, Racing, Swivel$199.99$139.99 -
Sale!
Professional LED Light Wired Gaming Headphones with Noise Cancelling Microphone
$29.99$19.99 Select optionsProfessional LED Light Wired Gaming Headphones with Noise Cancelling Microphone
SKU: N/A Categories: Electronics, Gaming, Gaming Headsets Tags: Cancelling, Gaming, Gaming Headphones with Noise Cancelling Microphone, gaming headset, Headphones, Headset, LED, Light, Mic, Microphone, Noise, Professional, Professional LED Light Wired Gaming Headphones, Professional LED Light Wired Gaming Headphones with Noise Cancelling Microphone, Wired, Wired Gaming Headphones, Wired Gaming Headphones with Noise Cancelling Microphone$29.99$19.99 -
Sale!
Gaming Desk with LED Lights USB Power Outlets and Charging Ports
$349.99$249.99 Select optionsGaming Desk with LED Lights USB Power Outlets and Charging Ports
SKU: N/A Categories: Computer Desk, Gaming, Gaming Desk Tags: and Charging Ports, Charging, Desk, Desks, Gaming, gaming desk with led lights, Gaming Desks with LED Lights, Home, LED, Lights, Monitor, Office, Outlets, Port, Power, Room, Stand, USB, USB Power Outlets, White, Workstation$349.99$249.99 -
Sale!
Wired Mixed Backlit Anti-Ghosting Gaming Keyboard
$99.99$79.99 Add to cartWired Mixed Backlit Anti-Ghosting Gaming Keyboard
Categories: Electronics, Gaming, Gaming Keyboards Tags: Antighosting, Backlit, Blue, brown, Gaming, Gaming Keyboard, gaming keyboards, gaming keyboards and mouse, Keyboard, Laptop, Switch, Wired, Wired Mixed Backlit Anti-Ghosting Gaming Keyboard, Wired Mixed Backlit Anti-Ghosting Gaming Keyboards, Wired Mixed Backlit Gaming Keyboard$99.99$79.99 -
Sale!
Wireless Bluetooth 5.3 ANC Noise Cancellation Hi-Res Over the Ear Headphones Headset
$119.99$59.99 Add to cartWireless Bluetooth 5.3 ANC Noise Cancellation Hi-Res Over the Ear Headphones Headset
Categories: Electronics, Gaming, Gaming Headsets Tags: 5.3 ANC Noise Cancellation Hi-Res Over the Ear Headphones Headset, ANC, Audio, Bluetooth, Cancellation, Ear, Earphone, gaming headset, Headphones, Headset, Hi-Res Over the Ear Headphones Headset, HiRes, Noise, Wireless, Wireless Bluetooth 5.3 ANC Noise Cancellation Hi-Res Headphones, Wireless Bluetooth 5.3 ANC Noise Cancellation Hi-Res Over the Ear Headphones Headset, Wireless Bluetooth 5.3 ANC Noise Cancellation Hi-Res Over the Ear Headphones Headsets$119.99$59.99 -
Sale!
Wired Sports Gaming Headset Earbuds with Microphone
$19.99$9.99 Select optionsWired Sports Gaming Headset Earbuds with Microphone
SKU: N/A Categories: Gaming, Gaming Headsets Tags: Accessories, Earbud, Earphone, Earphones, Gaming, gaming headset with microphone, Headphones, Headset, IOS, Microphone, Sports, Wired, Wired Sports Gaming Headset Earbuds, Wired Sports Gaming Headset Earbuds with Microphone, Wired Sports Headset Earbuds$19.99$9.99 -
Sale!
150W Universal Multi USB Fast Charger 16 Port MAX Charging Station
$49.99$29.99 Add to cart150W Universal Multi USB Fast Charger 16 Port MAX Charging Station
Categories: Charging Stations, Electronics Tags: 150W, 150W Charging Station, 150W Universal Multi USB Charging Station, 150W Universal Multi USB Fast Charger 16 Port MAX Charging Station, 150W Universal Multi USB Fast Charger 16 Port MAX Charging Stations, 150W Universal Multi USB MAX Charging Station, 16 Port MAX Charging Station, 3.5A, Charger, Charging, Fast, laptop charging stations, Max, Multi, Port, Stand, Station, Universal, USB$49.99$29.99
@ 5:20 it returns none. How to solve this issue?
thanks very much for this tutorial it helped a bit with my idea but you missed the most important part…
this requires for you to have your PC on and the script running all the time, right?
I am currently trying to find a way to deploy it so it can run on the web for free and just look at my email when it comes…
what do you think about that?
I noticed when I do the initial GET request, Amazon serves a CAPTCHA like page, blocking me from seeing the actual content. Is there a way around this? I wonder when the video was originally published if Amazon wasn't doing this at the time.
i end up with none…..
Hello Are you okay? I'm from Brazil, more precisely in the state of Paraná. I did a search on the GPT chat about python and he suggested your video about selenium, I have a question for you, is it possible to create something that identifies how long my employee is taking on average to serve my customers on whatsapp. Very grateful for your attention.
11:36
Why 587?
Can I put any number here?
Please reply…
What do I do if when I see the code from the inspector the element appears with its respective class and ids, but when I try to access it with beautiful soup.find it returns None, and I look in the source code by pressing (CTRL+U) and there That element doesn't appear, so I guess that's why the beatiful soup response None
What could I do there?
I'm NOT a programmer but I've been thinking about trying it out and this seems like a good beginner project. My question is once the program is coded, does it have to be uploaded anywhere to continue working? I don't understand how the program would still run while my computer is off if its just in vscode. Will it work while my computer is off because of what was imported? Sorry if my question doesn't make sense, I still haven't figured out how to efficiently ask programming questions.
This may be a stupid question but if you schedule the program to run, do you need your computer to be on for the scheduled program to run?
Hello, when I try to print title = soup.find(id=“productTitle) I keep getting a result of None. How do I fix this??
lets say i want to check the product stock – from "out of stock" to "instock"
how do i use the "if" for any changes on the tracked class/id?
i ve made the same thing
How to track full category of some item if price drops?
Did you have your leg up all the time while filming?
lol
hackerman was really funny and relatable
I''ve pip install requests and bs4 but it keeps saying modules are not found
0:01 Finally, something I qualify for
The symbol can be at the begining and also the length of the price is not guaranteed. So, instead we can use Regex like this:
import re //at the top of the file
convertedPrice = float(re.findall(r'd+.d+', price)[0])
re.findall accepts two parms => the first is the regex itself. d+ is find every number count, followed by . (dot) and any number count after the dot. In case we want the price after the dot.
and the second argument is the "price" that we extract using soup.find…..
and since re.findall returns an array but "float" doesn't accept array, we specify the first value of the array with [0]
Finally, convert the value to a number with float(….)
YOU’RE ROMANIAN?? I never knew, your american accent is very good, I had to do a double take when you showed us your email :O
good vid
Will the website be aware of the python checking that often? How could I have solution for different website with restrictions
Definitely trying this out
Amazon seems to deploy captcha anti-scraping techniques these days. Don't suppose there's an easy way around that?
did anyone else have the problem where the soup data was really short and had things like "if you want access go to some Subscription API" and "we want to make sure you're not a bot and you should enable cookies"
thank you so much for this tutorial! some things dont currently work (since this video is a few years old now and Amazon has changed your ability to scrape from them, it seems), but I still learned the basics about web scraping from this! I changed up my program to also send price differences from other websites too. Even the parts about 2-factor verification were useful and now its something extra I know! Thanks again!