LinkedIn profiles. Quick search through multiple files and add them to your CRM. Snel zoeken naar LinkedIn profielen en toevoegen aan CRM.

Search for LinkedIn profiles. Split them up into LinkedIn profile, gender, firstname, lastname, job position and company name. Now add them to your CRM at Ninox.

Zoek LinkedIn profielen. Split ze naar LinkedIn profiel, geslacht, voornaam, achternaam, functie en bedrijf. Nu toevoegen aan jouw CRM bij Ninox.

 

From this:

To

End result:

 

 

First part is how to search.

Second part is how to filter and tidy your results.

 

First part: how to search for LinkedIn profiles?

1. using a webcrawler and make use of a search engine

2. using services online (API's)

I am going to use a webcrawler to get results with a SERP (Search Engine).

You could also use one of the many API's from rapidapi or others. But most are POST requests and you can only pay by CreditCard over there.

And could be really expensive if you want to do a lot of searches.

Results are mostly in JSON. 

I like GET (because I can change the input myself in the URL). And it is easier to integrate in some software for me. I have no clue how Postman works ;)

 

I found some other services like https://serper.dev/

Results in JSON with a GET request.

I use Octoparse to handle the results (they have a free plan).

You could search Google with them.

The playground is nice.

You can pay by CreditCard, PayPal or Google Pay.

 

I tried with 50.000 credits for 50 USD.

Results were very fast and good.

Also I used multithreading (more requests at the same time, so not one after another, that was quicker to get the results in).

10 results cost me 1 credit.

20, 30, 40, 50 or 100 results cost me 2 credits.

If I choose 100 results, I can do 500 searches for 1 USD.

50 USD gives me 25.000 searches with the maximum of 100 results per search (maximum 2,5 million results in total).

 

 

 

 Also for Google Maps, if you want to search companies.

 

Or you could use a search engine and generate queries yourself.

If I choose duckduckgo, you can set up some parameters.

Search parameters.

https://duckduckgo.com/duckduckgo-help-pages/settings/params/

https://html.duckduckgo.com/html/?q={search terms, like word1+word2+word3}

https://html.duckduckgo.com/html/?q=linkedin.com+ceo+ninox

or for Dutch

https://html.duckduckgo.com/html/?q=linkedin.com+ceo+ninox&kl=nl-nl

 

I merge some words online.

A online tool is:

https://www.toptal.com/marketing/mergewords

But there are more (just search for it).

The merging is done in your browser. It get stuck with a lot of combinations.

For a lot of combinations you could better use a desktop tool. Like Keyword Combiner from Vovsoft.

https://vovsoft.com/software/keyword-combiner/

 

Toptal:

 

First fix the generated URLs.

 

 Change space for a plus sign.

I use TextPad (free to use) for that.

 

 

Copy paste the URLs in a webcrawler.

I use Netpeak Spider in this example.

Only interested in external links.

 

 

Some settings in Netpeak Spider.

 No cookies are stored in Duckduckgo, so you can turn it off.

 

 The User agent is important, if you choose the 'wrong' one. Results will be unreadable.

 

 

 

 

You can choose to work with your own IP address (adjust the max amount of threads).

Or work from behind a VPN and let the tool work over the VPN.

Or work with proxies.

 

To see the results and filter.

 

 

 You could do an export on this filtered results.

Output in CSV.

 

Per page it shows 1 million results.

If you filter it, you see the filtered out results of that million.

So if you have 3 million results, you have 3 pages.

When filtered, you still have 3 pages with filtered results.

You have to export per page.

 

You could also export from the main window.

 You can also export in a different way.

External links (XL)

 

 

All unique URLs & anchors (XL)

 

Second part: Clean up the found results (split into LinkedIn profile, full name, gender, first name, last name, job position, company).

Using Rons Data Edit for that.

 

 You might clean up your data.

I use Rons Data Edit for that. A great CSV editor.

https://www.ronsplace.ca/Products/RonsDataEdit

 

I am only interested in URLs containing Linkedin.com

And the anchor must contain text.

 

 I filter more.

Like only linkedin.com/in/ for personal profiles

or linkedin.com/company/ for company profiles.

 

Then you save that filtered view.

Otherwise it saves the whole (unfiltered) file.

 

You don't want to have duplicate content.

Easy deleting of duplicate rows.

 

As a test, I added some files containing LinkedIn profiles.

Didn't do much of filtering and deleting duplicate rows (was a quick test).

I split up the big file into smaller files (containing 100.000 rows each).

Now you can search in the results.

 

You could even do some more filtering on the file.

I want to only have personal linkedin profiles (linkedin.com/in/) and finding a person (anchor - and | Linkedin)

 

 

 You can split up the columns to get full name, job position and company.

 

 

 

 

 

 Some results don't have a job position.

I first save the file (save view).

And then open the filtered file and continue to filter more.

It is a bit easier, because the file is smaller now (so filtering and splitting up works faster).

We filter and split again.

 Filtering on text (only a company name and no job position).

 

 We make a new column with company name.

 

 Filtering again

 

 I cut the content and paste it into column Company name.

 Now you do the other part.

 

 

 

 The result:

Some cells did split up the wrong way.

I fix that manually, it is not that much. Just cut and paste text.

 We remove the space from beginning in a cell.

 

 

 We delete all beginning with | LinkedIn (column Company name).

 

 Delete the new column.

 Splitting up the full name into first name and last name.

I have to think about something how to split on 'tussenvoegsels' like Pieter de Bruin 'de'. In the Netherlands we have 333 sorts of that (Belgium, Germany, South-Africa and many other countries have it too).

 

To add gender we could use genderize.io

 We make a new column with gender.

 Use the Fill Web API.

 

 Press Fetch Mappings

 Now you can determine which Source Paths you want to add.

 I choose only gender.

 Press Fill

 It gets filled within seconds.