Auto news on Youtube Mar 2 2018

In this video you will learn how to use PHP CURL, we are going to do three

different examples in the first one we will scrape the results from Google in

the second one from my personal website codingpassiveincome.com and then in the

third example I'm going to show how to log into some website so stay with me :)

Hey what's up guys Senaid here codingpassiveincome.com the place where

I help others to become a web developer much easier and faster than they will do

it on their own so if that's something that interests you, consider subscribing -->

So for those of you that don't know what curl is basically we use it

mainly from PHP when we want to communicate with different web sites so

if you want to get an information from some website we usually use curl because

of its speed so in the first example that I mentioned in the introduction I

said that we are going to scrape the results from the Google so first thing

as you can see if we go to the Google and then try to search for something

let's say test whatever you can see that they always use Google dot and then the

name from the country that you are coming it mostly then search and then

here they have a pendant of a lot of different things but the one that we are

interested in is this query basic this query and then what you are searching

for so basically we can delete all the other things like this and then if we

make a test let's say another test you can see now that it is working and that's

what we are going to do so that's the URL that we want to scrape so I will

open PHP tags and first that we need to do is to actually in it call so I'll

create variable curl handler like shortcut and then I'll say curl_init

and that's it and now for this curl we always need to add some options

depending on what you are going to do so you will see through this

examples what are the different options that you have so let's say the first one

that you are going to do so as you can see first parameter is the

curl handler so just right here your variable name and then the option that

you want to set so let's say this time we will just set URL and then here we

need to specify the URL okay so now our URL is this one okay and then here are

the curl we just need to specify the keyword that

we are searching for so let's say Senaid Bacinovic

and then I usually use plus just like URL encoding so the next that we

need to set for this example is and then follow location which means that if

Google redirects us from this URL to some other that they want mainly they do

it if you are choosing the wrong domain so we yes we want to follow location and

then we also one that we get the return from in Google so it says return

transfer at one so here I set one you can set true or 0 or false

so it's saying so true or one and false for zero whatever you want okay

and I think that's it let's make a test so now to make a test

we just need to declare a variable where we will store whatever Google returns so

you'll say call and then execute and then we want to choose this curl Handler

and of course you are going just to close this handler

close the connection okay and then let's just the echo response so let's

see what will happen okay I'll hit refresh and it works here

we go so as you can see now we have get all

the results for this keyword and now if we do the same thing here and search for

me here you guys can see Twitter linkedin and in my website etc etc so this is how

to get but now the thing is let's see how we are going to parse for example

all texts from this so that you get only them because we are scraping the google

for some reason and now there are different ways on how we can scrape some

website I mean how we can analyze the return on

response HTML document and go through it and for that reason I want to show you

how to use simple dom parser okay it's one very lightweight but very powerful

library so we will just dowonload it okay latest version as you can see is

just few kilobytes it's very small but very very powerful and with this library

you basically are able to search through this document just like you will do in

jQuery which is really powerful powerful okay so just extract here delete the

archive and this file there's any includes so I don't think that we need

any of those we doesn't need this part but we will see

and that's it so at the top I will include simple HTML dm PHP okay and

then here of course we need to utilize so let's say i will create HTML variable out there is no variable and

then I will just utilize this simple HTML Dom and then I'm just going to load

the response from Google and then let's see will we get any error so I hit

refresh and it's empty okay I'm not sure why so let's try to

say he sorry it's okay we didn't print out anything so that's

why it's empty that means that there isn't any error and then now let's try

to search for the titles so here let's see our source code this is from the

Google and then let's try to search for the one and if we go again to the Google

you can see that first one is for the Twitter so let's find it so say

twitter.com okay and here we go so you can see here that we have one link and

then in that link we also have the title that we are looking for like here so we

can see that every link is starting with href and then /url question mark and

then some URL so we can say something like here for each and then let's say

HTML fine and then we will do a selector like

we will use in jQuery so say ref is starting with then /url question mark

and then let's say as link we want to get to print out link and then I will

just say inner or maybe plain text so you have different options here to

extract and you can find it if you go here and then maybe go to the online

document about simple dawn and then here of course you have explanation on

different things and how you can use it okay so just read the documentation if

you want to learn more okay and let's make a test for our case scenario and

here we go so maybe let's just add a new line okay and here just in six link it

linkedin twitter programmer and blogger senaid and as you can see here we have something

weird so I'm not sure what this

module this is on Bosnian language and let's try to find here so it's here and

let's see we can say that if we have found URL with this web page that Google

user contact.com we don't want to include it so maybe we can say here if

string position and then let's say in our link

then let's see if we find where catch dot Google etc equally false

that means if we don't find your just print out this text so let's see and

it's gone so now we have got only the results about me in this search which is

what you were looking for and now if you know how to use this simple HTML Dom

parser you will know how to extract anything from that page okay so that is

the first example so let's go to another one in the next one we are going to try

to get the informations from my personal website so this is my website and here

are all the tutorials that I have there's a lot of them and let's try to

get and see what will happen so I will just paste the link here so let's we will use a cell

and we will just echo the response and here will will say exit just test this part here

okay and now if you see there isn't any tutorial this is just something about my

website so the problem is that if you go to the f12 hit refresh you can see that

I'm dynamically loading all the tutorials so if we may maybe go down and

click on the load more you can see here that I'm going to make

and Ajax call and then with a ajax call I'm going to load more tutorials

you know and if we do it again same thing will happen ok so now it means

that we will need to execute the post request to this URL in order to get more

tutorials and for that reason we will need to modify this script a little bit

so I'll say ajax X dot PHP and now here you can see that in the phone data

I'm sending four different things so we will need to send them too so let's say

here I will create one variable post files and then create an array okay so

this is associative array and it means that we need to do it like this

can be boring sometimes but okay and now as you can see I have some start

variable and some limit okay so let's say I am starting from zero and want to

get 10 tutorials okay and here in our curl now we need to say to the curl

that we are going to make a post request so for that reason we will say here curl

Handler and then call off and then let's say post files will be our pots files

but I will just use and build HTTP query so say postfiles okay so I will duplicate

quickly remove this and then say we are making post and setting it to the one

okay and let's make a test now so refresh and as you can see now we are

getting only the posts from my website and if we go to control you there's a

lot of HTML so maybe let's say a echo free

so the problem is that this is not nicely sorted because you can see here

we have Jason so I will just say here that you'll say JSON decode and then

here I'll say response race which is this here okay so let's now refresh

and here we go so now here you can see that we have each time different columns

and in each column we are getting different tutorials okay and now we can

see here that for every tutorial I do have one h2 two box title and in this

title they will have a link and that link we are going to have a title of

this tutorial so again we are going to use our simple HTML Dom okay so here

I'll say response recent and then here we will search for h2 and then we can

just copy the class from here

and then we will remove this part and say a echo link plaintext okay and let's you

here we go so as you can see now even from my own website I

have got all the tutorials but now the difference is that we needed to add post

and post fields to this call or it wouldn't work and now the last thing

that I want to do for this tutorial is to show you how to log in to some

website so let's say we will again use my website as an example so in my website I

do have a special members dashboard those are the people that have bought

some course from me and now this is the login page to get an access to some of

those courses as you see now here this drop down you are able to select one of

the courses okay and let's say we are going to use this one and then we need

to provide email and the password okay so I have set some temporary password

enter and if we log end it will show the message and login so now it's obvious

that this is done over jQuery AJAX because we didn't refresh page and we

have logged in so for that reason we will need to know

what are the information that you are standing and we are sending those so of

course if you know that it is a AJAX code then just go to the source code and see

what's happening there and as you can see here the code is pretty simple and

in Ajax we are sending the for informations which is check login

as a key email password and the course ID and course ID we are getting from

this select and we just get an option so let's do it I will just get this or maybe

we can just quickly write password and then course ID so email is mine okay

that's what we enter and then course ID is for this one is number one okay and

that's it so I will save and then we need to set our URL okay so URL we were send

information is login.php which is this URL so how I will just copy okay and that's it

so let's again see what will happen and here I will just say X

says sto could not be connected to string that's okay let's remove this

at moment and here it goes as you can see we have got JSON response status is 1 and the

first lesson is number one which means that we have successfully logged in but

the thing is that we want to see the members dashboard and not only this

message so that means that we need to store somewhere the cookie informations

and for that reason there is another option for the call okay call handler

call and then we need cookie and then we need to set the path to that file so

let's say I'll just say in my case cookie dot txt okay and now here our

I will just say decode this and then I am going to check so see if response status that

means that we are logged in and we are going to make another call handler

another call to get this let's see where will it redirect us as we going okay so

if we are logged in we are going to make a test and try to get this page so here

I will just remove this because I don't want to close this curl that's very

important and then here I will just paste everything and then post and post fields

we don't need next time we just need this URL

okay okay and this ID is basically from response

first lesson okay and of course you don't need JSON decode because now we are

turning the entire HTML okay and let's not try to get everything from this

responds to seem to be logged in successfully so here I'll say refresh and

here we go as you can see now we have from the left side in menu that you

should have like this and we have here some content for the first lesson

whatever it might be and if we refresh here is the source code that we are

getting okay which is amazing we did correctly this part and now the next

thing that I want to do is let's actually try to get the menu items from

this dashboard so let's see how we can get this those and as you can see here

the each time have this link that which href is dating with view lesson

dot PHP so we can say here right and then we'll just say starts with view

lesson dot PHP IDE okay and let's make up test

object what is the problem here we should flow the reason to just need this

response and as you can see now we have all the links from our page so guys

pretty much that's it I hope that you have enjoyed this video and if you do

please like it and share with your friends and of course if you have any

question doubts or anything please ask in the comments below take care :)