When scraping websites for lead generation, finding the "About Us" page offers invaluable insights. Typically, these pages provide critical information including the business's history, values, leadership team, and contact details.
Here are some of the most common URL structures used for "About Us" pages:
<ul>
<li>/about</li>
<li>/about-us</li>
<li>/about_me (for personal sites or blogs)</li>
<li>/aboutus</li>
<li>/company/about</li>
<li>/company</li>
<li>/our-story</li>
<li>/our-company</li>
<li>/who-we-are</li>
<li>/meet-the-team</li>
<li>/about-our-company</li>
<li>/about-company</li>
<li>/aboutus.html</li>
<li>/about/index.html</li>
</ul>
These variations are widely used across different websites and industries.
<h1 id="user-content-how-to-automatically-follow-about-us-pages-when-scraping"><a href="#how-to-automatically-follow-about-us-pages-when-scraping" aria-hidden="true" tabindex="-1"></a>How to automatically follow "About us" pages when scraping?</h1>
With this list of URL paths, it's easy to scrape any website and follow the "About" link in order to get data.
The best examples of data to extract are:
<ul>
<li>Email addresses</li>
<li>LinkedIn Company pages</li>
<li>LinkedIn People profiles</li>
<li>Phone numbers</li>
<li>Social Links (Instagram, Facebook, Youtube, etc.)</li>
<li>Texts that describe what a company does</li>
</ul>
This is what Datablist <a href="/enrichments/scrape-texts-linkedin-email-instagram-links-webpage-bulk">Smart Scraper</a> does!
You can define a list of websites and the Smart Scraper will visit each website to extract data from the defined webpage and the website about page. <a href="/enrichments/scrape-texts-linkedin-email-instagram-links-webpage-bulk">Click here to learn more about the Smart Scraper</a>.
<img src="/learn_images/scraping/about-us-paths/outputs.png#2355x1361" alt="Smart Scraper">

When scraping websites for lead generation, finding the "About Us" page offers invaluable insights. Typically, these pages provide critical information including the business's history, values, leadership team, and contact details.

Here are some of the most common URL structures used for "About Us" pages:

- /about
- /about-us
- /about_me (for personal sites or blogs)
- /aboutus
- /company/about
- /company
- /our-story
- /our-company
- /who-we-are
- /meet-the-team
- /about-our-company
- /about-company
- /aboutus.html
- /about/index.html

These variations are widely used across different websites and industries.

# How to automatically follow "About us" pages when scraping?

With this list of URL paths, it's easy to scrape any website and follow the "About" link in order to get data.

The best examples of data to extract are:

- Email addresses
- LinkedIn Company pages
- LinkedIn People profiles
- Phone numbers
- Social Links (Instagram, Facebook, Youtube, etc.)
- Texts that describe what a company does

This is what Datablist [Smart Scraper](/enrichments/scrape-texts-linkedin-email-instagram-links-webpage-bulk) does!
You can define a list of websites and the Smart Scraper will visit each website to extract data from the defined webpage and the website about page. [Click here to learn more about the Smart Scraper](/enrichments/scrape-texts-linkedin-email-instagram-links-webpage-bulk).

![Smart Scraper](/learn_images/scraping/about-us-paths/outputs.png#2355x1361)

<aside class="text-center mb-5">
 <div class="mb-2">
 <a href="https://app.datablist.com/" className="btn btn-primary btn-large" target="_blank">
 Open Datablist
 </a>
 </div>
 No signup required
</aside>

The most common URL paths for "About Us" pages

When scraping websites for lead generation, finding the &ampquot;About Us&ampquot; page offers invaluable insights. Typically, these pages provide critical information including the business&amp#39;s history, values, leadership team, and contact details.
Here are some of the most common URL structures used for &ampquot;About Us&ampquot; pages:
/about
/about-us
/about_me (for personal sites or blogs)
/aboutus
/company/about
/company
/our-story
/our-company
/who-we-are
/meet-the-team
/about-our-company
/about-company
/aboutus.html
/about/index.html

These variations are widely used across different websites and industries.
How to automatically follow &ampquot;About us&ampquot; pages when scraping?
With this list of URL paths, it&amp#39;s easy to scrape any website and follow the &ampquot;About&ampquot; link in order to get data.
The best examples of data to extract are:
Email addresses
LinkedIn Company pages
LinkedIn People profiles
Phone numbers
Social Links (Instagram, Facebook, Youtube, etc.)
Texts that describe what a company does

This is what Datablist Smart Scraper does!
You can define a list of websites and the Smart Scraper will visit each website to extract data from the defined webpage and the website about page. Click here to learn more about the Smart Scraper.
Smart Scraper


In lead generation, for businesses targeting Shopify stores, extracting the country code from a list of Shopify URLs can be a game-changer.
Hopefully, the country code information is quickly available in the source code of the Shopify stores.
In this article, you will learn:
<ul>
<li><a href="#how-to-find-the-country-code-for">To find the country code manually for a Shopify store.</a></li>
<li><a href="#how-to-extract-country-codes-for-a-list-of-shopify-stores---in-bulk">Extract the country codes for a list of Shopify stores in bulk.</a></li>
</ul>
<h2 id="user-content-how-to-find-the-country-code-for-a-shopify-store-manually"><a href="#how-to-find-the-country-code-for-a-shopify-store-manually" aria-hidden="true" tabindex="-1"></a>How to find the country code for a Shopify Store manually</h2>
<img src="/learn_images/scraping/extract-country-codes/view-source-code-shopify.png#2209x1199" alt="Open Shopify source code">
<img src="/learn_images/scraping/extract-country-codes/search-country-code.png#1472x1302" alt="Search &#x22;Shopify.country&#x22;">
<h2 id="user-content-how-to-extract-country-codes-for-a-list-of-shopify-stores---in-bulk"><a href="#how-to-extract-country-codes-for-a-list-of-shopify-stores---in-bulk" aria-hidden="true" tabindex="-1"></a>How to extract country codes for a list of Shopify Stores - In Bulk</h2>
Now, to extract country codes in bulk, you need to automate this process.
To do so, you can use Datablist Bulk Scraper. Datablist is a powerful <a href="/use-cases/lead-list-building">Leads Relationship Management</a> tool.
First, import your list of Shopify stores in Datablist <a href="/csv-editor">CSV Editor</a>.
Then, click on the "Enrich" button in the header.
<img src="/learn_images/scraping/extract-country-codes/import-list-stores.png#2473x1480" alt="Import List of Shopify Stores">
Select the <a href="/enrichments/scrape-urls-css-selectors-regex-bulk">Bulk Scraper</a> enrichment. The Bulk Scraper will open each store using the Shopify URL and extract the country code from the HTML source code.
<img src="/learn_images/scraping/extract-country-codes/bulk-scraper-enrichment.png#2337x1042" alt="Select Bulk Scraper">
In the scraper configuration, scroll to the "Regex #1" setting. And write the following regular expression:
<code>Shopify\.country\s=\s"(\w+)";</code>
<img src="/learn_images/scraping/extract-country-codes/configure-regex.png#2256x1308" alt="Configure Input with Regular Expression">
Then, continue to configure the outputs. Create new properties to store the Regex Text #1.
You can also add the URL Scraping Status output to know when a store is no longer online.
<img src="/learn_images/scraping/extract-country-codes/scraping-outputs.png#2258x1370" alt="Configure Scraping Outputs">
Then run the Bulk Scraper enrichment on your first 10 stores. If the results are good, process the whole list.
To finish, clean unreachable stores to keep only up-to-date Shopify leads.
<img src="/learn_images/scraping/extract-country-codes/shopify-stores-country-code-results.png#1646x1146" alt="Shopify Stores Country Code List">

In lead generation, for businesses targeting Shopify stores, extracting the country code from a list of Shopify URLs can be a game-changer.

Hopefully, the country code information is quickly available in the source code of the Shopify stores.

In this article, you will learn:

- [To find the country code manually for a Shopify store.](#how-to-find-the-country-code-for)
- [Extract the country codes for a list of Shopify stores **in bulk**.](#how-to-extract-country-codes-for-a-list-of-shopify-stores---in-bulk)


## How to find the country code for a Shopify Store manually


![Open Shopify source code](/learn_images/scraping/extract-country-codes/view-source-code-shopify.png#2209x1199)

![Search "Shopify.country"](/learn_images/scraping/extract-country-codes/search-country-code.png#1472x1302)


## How to extract country codes for a list of Shopify Stores - In Bulk

Now, to extract country codes in bulk, you need to automate this process.

To do so, you can use Datablist Bulk Scraper. Datablist is a powerful [Leads Relationship Management](/use-cases/lead-list-building) tool.

First, import your list of Shopify stores in Datablist [CSV Editor](/csv-editor).

<aside class="text-center mb-5">
 <div class="mb-2">
 <a href="https://app.datablist.com/auth/signup" className="btn btn-primary btn-large" target="_blank">
 Open Datablist
 </a>
 </div>
 Sign up for free
</aside>

Then, click on the "Enrich" button in the header.

![Import List of Shopify Stores](/learn_images/scraping/extract-country-codes/import-list-stores.png#2473x1480)

Select the [**Bulk Scraper**](/enrichments/scrape-urls-css-selectors-regex-bulk) enrichment. The Bulk Scraper will open each store using the Shopify URL and extract the country code from the HTML source code.

![Select Bulk Scraper](/learn_images/scraping/extract-country-codes/bulk-scraper-enrichment.png#2337x1042)

In the scraper configuration, scroll to the "Regex #1" setting. And write the following regular expression:

```Shopify\.country\s=\s"(\w+)";```

![Configure Input with Regular Expression](/learn_images/scraping/extract-country-codes/configure-regex.png#2256x1308)

Then, continue to configure the outputs. Create new properties to store the **Regex Text #1**.

You can also add the **URL Scraping Status** output to know when a store is no longer online.

![Configure Scraping Outputs](/learn_images/scraping/extract-country-codes/scraping-outputs.png#2258x1370)

Then run the Bulk Scraper enrichment on your first 10 stores. If the results are good, process the whole list.

To finish, clean unreachable stores to keep only up-to-date Shopify leads.

![Shopify Stores Country Code List](/learn_images/scraping/extract-country-codes/shopify-stores-country-code-results.png#1646x1146)

Extracy country codes from Shopify stores

In lead generation, for businesses targeting Shopify stores, extracting the country code from a list of Shopify URLs can be a game-changer.
Hopefully, the country code information is quickly available in the source code of the Shopify stores.
In this article, you will learn:
To find the country code manually for a Shopify store.
Extract the country codes for a list of Shopify stores in bulk.

How to find the country code for a Shopify Store manually
Open Shopify source code
Search &ampquot;Shopify.country&ampquot;
How to extract country codes for a list of Shopify Stores - In Bulk
Now, to extract country codes in bulk, you need to automate this process.
To do so, you can use Datablist Bulk Scraper. Datablist is a powerful Leads Relationship Management tool.
First, import your list of Shopify stores in Datablist CSV Editor.
Then, click on the &ampquot;Enrich&ampquot; button in the header.
Import List of Shopify Stores
Select the Bulk Scraper enrichment. The Bulk Scraper will open each store using the Shopify URL and extract the country code from the HTML source code.
Select Bulk Scraper
In the scraper configuration, scroll to the &ampquot;Regex #1&ampquot; setting. And write the following regular expression:
Shopify\.country\s=\s&ampquot;(\w+)&ampquot;;
Configure Input with Regular Expression
Then, continue to configure the outputs. Create new properties to store the Regex Text #1.
You can also add the URL Scraping Status output to know when a store is no longer online.
Configure Scraping Outputs
Then run the Bulk Scraper enrichment on your first 10 stores. If the results are good, process the whole list.
To finish, clean unreachable stores to keep only up-to-date Shopify leads.
Shopify Stores Country Code List


Here is a quick JavaScript snippet to extract all URLs from a webpage fast with Google Chrome Developer Tools. No Browser Extension is required!
The JavaScript code generates a list of URLs in CSV format with the anchor texts, and a boolean to know if the URL is internal or external to the current website. Just copy-paste the results into Google Sheets or with the Datablist <a href="/csv-editor">CSV editor</a>.
<blockquote>
Notes: The code works also with Firefox, Safari, etc.
</blockquote>
<h2 id="user-content-step-1-run-javascript-code-in-google-chrome-developer-tools"><a href="#step-1-run-javascript-code-in-google-chrome-developer-tools" aria-hidden="true" tabindex="-1"></a>Step 1: Run JavaScript code in Google Chrome Developer Tools</h2>
<ul>
<li>Open Google Chrome Developer Tools with <code>Cmd + Opt + i</code> (Mac) or <code>F12</code> (Windows). And click on the <code>Console</code> tab.</li>
<li>Copy-Paste the following JavaScript code and press <code>Enter</code>.</li>
</ul>
<pre><code>const results = [
 ['Url', 'Anchor Text', 'External']
];
var urls = document.getElementsByTagName('a');
for (urlIndex in urls) {
 const url = urls[urlIndex]
 const externalLink = url.host !== window.location.host
 if(url.href &#x26;&#x26; url.href.indexOf('://')!==-1) results.push([url.href, url.text, externalLink]) // url.rel
}
const csvContent = results.map((line)=>{
 return line.map((cell)=>{
 if(typeof(cell)==='boolean') return cell ? 'TRUE': 'FALSE'
 if(!cell) return ''
 let value = cell.replace(/[\f\n\v]*\n\s*/g, "\n").replace(/[\t\f ]+/g, ' ');
 value = value.replace(/\t/g, ' ').trim();
 return `"${value}"`
 }).join('\t')
}).join("\n");
console.log(csvContent)
</code></pre>
<img src="/learn_images/scraping/extract-urls-webpage/copy-js-code.png#1938x694" alt="Open Google Developer Tools">
<h2 id="user-content-step-2-copy-paste-exported-urls-into-a-csv-file-or-spreadsheet-tools"><a href="#step-2-copy-paste-exported-urls-into-a-csv-file-or-spreadsheet-tools" aria-hidden="true" tabindex="-1"></a>Step 2: Copy-paste exported URLs into a CSV file or spreadsheet tools</h2>
The JavaScript code will extract all URLs from the webpage with the following values:
<ul>
<li>URL - The link URL</li>
<li>Anchor Text - The label associated with the link. Called "Anchor Text".</li>
<li>External - A boolean (<code>TRUE</code>, <code>FALSE</code>)</li>
</ul>
The result is printed on the console using <a href="/learn/csv/csv-meaning">CSV format</a>. You can copy-paste the result into spreadsheet tools or with our <a href="/csv-editor">online CSV editor</a>. Or create a new text file with your text editor and save the file with a <code>.csv</code> extension.
Google automatically adds a <code>Copy</code> button when you get a lot of results.
<img src="/learn_images/scraping/extract-urls-webpage/copy-results.png#1790x664" alt="Open Google Results">
<h2 id="user-content-step-3-filter-csv-data-to-get-relevant-links"><a href="#step-3-filter-csv-data-to-get-relevant-links" aria-hidden="true" tabindex="-1"></a>Step 3: Filter CSV data to get relevant links</h2>
To filter quickly your links, open Datablist <a href="/csv-editor">CSV editor</a>. Create a collection and paste your CSV data. Check the <a href="#example-with-google-results">Google</a> and <a href="#example-with-amazon-results">Amazon</a> examples below for a more step-by-step guide.
<img src="/learn_images/scraping/extract-urls-webpage/create-collection.png#1064x668" alt="Create new collection">
Select 'First row contains header' and click on the "+" button for each column to import them.
<img src="/learn_images/scraping/extract-urls-webpage/paste-results.png#2537x1346" alt="Paste results">
Then you will be able to use text search or filter on a specific property.
You can use Datablist to <a href="/features/combine-csv-files">consolidate all your CSV data</a>. And Datablist comes with an algorithm for <a href="/features/duplicates-remover">deduplication</a> to have a clean list or URLs.
<h2 id="user-content-example-with-google-results"><a href="#example-with-google-results" aria-hidden="true" tabindex="-1"></a>Example with Google Results</h2>
This is a step-by-step example with the Google results page. We want to extract all external links from a Google search result.
<h3 id="user-content-step-1-search-for-a-google-term-that-you-want-to-extract-links"><a href="#step-1-search-for-a-google-term-that-you-want-to-extract-links" aria-hidden="true" tabindex="-1"></a>Step 1: Search for a Google term that you want to extract links.</h3>
A nice tip is to add the <code>&#x26;num=100</code> parameter to the URL to force Google into showing 100 results per page.
<img src="/learn_images/scraping/extract-urls-webpage/google/google-num-param.png#1696x458" alt="Force 100 results">
<img src="/learn_images/scraping/extract-urls-webpage/google/step-1-google-results.png#1934x1300" alt="Open Google Results">
<h3 id="user-content-step-2-copy-paste-the-code-and-run-it"><a href="#step-2-copy-paste-the-code-and-run-it" aria-hidden="true" tabindex="-1"></a>Step 2: Copy-paste the code and run it</h3>
Open the <code>console</code> in Google Developer Tools, paste the JavaScript code and press <code>Enter</code>.
<img src="/learn_images/scraping/extract-urls-webpage/google/step2-copy-paste-code.png#2532x1235" alt="Copy-paste code">
Copy the results. Select the line and <code>ctrl + c</code> or use the <code>Copy</code> button when Chrome shows it.
<img src="/learn_images/scraping/extract-urls-webpage/google/step3-run-code.png#2532x1344" alt="Run JavaScript code">
<h3 id="user-content-step-3-filter-results"><a href="#step-3-filter-results" aria-hidden="true" tabindex="-1"></a>Step 3: Filter results</h3>
Open Datablist <a href="/csv-editor">CSV editor</a>, create a collection, and paste the copied URLs.
<blockquote>
Note: Don't forget to select the option <code>First row contains headers</code> and to click the "+" button as shown.
</blockquote>
<img src="/learn_images/scraping/extract-urls-webpage/google/step4-create-collection-datablist.png#2537x1346" alt="Copy-Paste results in Datablist">
Once imported. Click "Filter on property" in the "External" column.
<img src="/learn_images/scraping/extract-urls-webpage/google/step5-filter-1.png#2760x1750" alt="Create Filters">
And filter items with <code>External</code> to <code>TRUE</code> and <code>Url</code> that does not contain <code>google</code>. Check our guide on <a href="/how-to/clean-web-scraping-data-online">how to clean your scraped data</a>
<img src="/learn_images/scraping/extract-urls-webpage/google/step5-filter-2.png#2533x1355" alt="Filter URLs outside Google">
And here are your external links:
<img src="/learn_images/scraping/extract-urls-webpage/google/step6-results.png#2371x1432" alt="Google Links results">
<h2 id="user-content-example-with-amazon-results"><a href="#example-with-amazon-results" aria-hidden="true" tabindex="-1"></a>Example with Amazon Results</h2>
This is a step-by-step example with the Amazon products page. We want to extract all links from this page.
<img src="/learn_images/scraping/extract-urls-webpage/amazon/amazon-speakers-results.png#1926x1588" alt="Amazon products listing">
Open the Chrome Developer Tools:
<img src="/learn_images/scraping/extract-urls-webpage/amazon/amazon-open-developer-tools.png#1930x1650" alt="Open Chrome Developer Tools">
Copy-paste the JavaScript code:
<img src="/learn_images/scraping/extract-urls-webpage/amazon/amazon-copy-paste-code.png#2162x1830" alt="Copy-Paste code">
And run it by pressing <code>Enter</code>:
<img src="/learn_images/scraping/extract-urls-webpage/amazon/amazon-run-code.png#2162x1830" alt="Run code">
With Datablist <a href="/csv-editor">CSV editor</a>, create a collection.
<img src="/learn_images/scraping/extract-urls-webpage/amazon/amazon-create-collection.png#2185x1098" alt="Create collection">
Paste your scrapped URLs:
<img src="/learn_images/scraping/extract-urls-webpage/amazon/import-from-pasting.png#2195x1202" alt="Paste scrapped URLs">
Your scrapped URLs are listed on the tool. Filter the links, <a href="/features/duplicates-remover">deduplicate the results</a>, and <a href="/how-to/clean-web-scraping-data-online">clean your scraped data</a>.
<img src="/learn_images/scraping/extract-urls-webpage/amazon/amazon-results.png#2643x1366" alt="Scrapped Amazon URLs">


Here is a quick JavaScript snippet to extract all URLs from a webpage fast with Google Chrome Developer Tools. No Browser Extension is required!

**The JavaScript code generates a list of URLs in CSV format with the anchor texts, and a boolean to know if the URL is internal or external to the current website. Just copy-paste the results into Google Sheets or with the Datablist [CSV editor](/csv-editor).**

> **Notes**: The code works also with Firefox, Safari, etc.

## Step 1: Run JavaScript code in Google Chrome Developer Tools

- Open Google Chrome Developer Tools with `Cmd + Opt + i` (Mac) or `F12` (Windows). And click on the `Console` tab.
- Copy-Paste the following JavaScript code and press `Enter`.

```
const results = [
    ['Url', 'Anchor Text', 'External']
];
var urls = document.getElementsByTagName('a');
for (urlIndex in urls) {
    const url = urls[urlIndex]
    const externalLink = url.host !== window.location.host
    if(url.href && url.href.indexOf('://')!==-1) results.push([url.href, url.text, externalLink]) // url.rel
}
const csvContent = results.map((line)=>{
    return line.map((cell)=>{
        if(typeof(cell)==='boolean') return cell ? 'TRUE': 'FALSE'
        if(!cell) return ''
        let value = cell.replace(/[\f\n\v]*\n\s*/g, "\n").replace(/[\t\f ]+/g, ' ');
        value = value.replace(/\t/g, ' ').trim();
        return `"${value}"`
    }).join('\t')
}).join("\n");
console.log(csvContent)
```

![Open Google Developer Tools](/learn_images/scraping/extract-urls-webpage/copy-js-code.png#1938x694)


## Step 2: Copy-paste exported URLs into a CSV file or spreadsheet tools

The JavaScript code will extract all URLs from the webpage with the following values:

- **URL** - The link URL
- **Anchor Text** - The label associated with the link. Called "Anchor Text".
- **External** - A boolean (`TRUE`, `FALSE`)

The result is printed on the console using [CSV format](/learn/csv/csv-meaning). You can copy-paste the result into spreadsheet tools or with our [online CSV editor](/csv-editor). Or create a new text file with your text editor and save the file with a `.csv` extension.

Google automatically adds a `Copy` button when you get a lot of results.

![Open Google Results](/learn_images/scraping/extract-urls-webpage/copy-results.png#1790x664)


## Step 3: Filter CSV data to get relevant links

To filter quickly your links, open Datablist [CSV editor](/csv-editor). Create a collection and paste your CSV data. Check the [Google](#example-with-google-results) and [Amazon](#example-with-amazon-results) examples below for a more step-by-step guide.

![Create new collection](/learn_images/scraping/extract-urls-webpage/create-collection.png#1064x668)

Select 'First row contains header' and click on the "+" button for each column to import them.

![Paste results](/learn_images/scraping/extract-urls-webpage/paste-results.png#2537x1346)

Then you will be able to use text search or filter on a specific property.
You can use Datablist to [consolidate all your CSV data](/features/combine-csv-files). And Datablist comes with an algorithm for [deduplication](/features/duplicates-remover) to have a clean list or URLs.


## Example with Google Results

This is a step-by-step example with the Google results page. We want to extract all external links from a Google search result.

### Step 1: Search for a Google term that you want to extract links.

**A nice tip is to add the `&num=100` parameter to the URL to force Google into showing 100 results per page.**

![Force 100 results](/learn_images/scraping/extract-urls-webpage/google/google-num-param.png#1696x458)

![Open Google Results](/learn_images/scraping/extract-urls-webpage/google/step-1-google-results.png#1934x1300)


### Step 2: Copy-paste the code and run it

Open the `console` in Google Developer Tools, paste the JavaScript code and press `Enter`.

![Copy-paste code](/learn_images/scraping/extract-urls-webpage/google/step2-copy-paste-code.png#2532x1235)

Copy the results. Select the line and `ctrl + c` or use the `Copy` button when Chrome shows it.

![Run JavaScript code](/learn_images/scraping/extract-urls-webpage/google/step3-run-code.png#2532x1344)

### Step 3: Filter results

Open Datablist [CSV editor](/csv-editor), create a collection, and paste the copied URLs.

> **Note**: Don't forget to select the option `First row contains headers` and to click the "+" button as shown.

![Copy-Paste results in Datablist](/learn_images/scraping/extract-urls-webpage/google/step4-create-collection-datablist.png#2537x1346)

Once imported. Click "Filter on property" in the "External" column.

![Create Filters](/learn_images/scraping/extract-urls-webpage/google/step5-filter-1.png#2760x1750)


And filter items with `External` to `TRUE` and `Url` that does not contain `google`. Check our guide on [how to clean your scraped data](/how-to/clean-web-scraping-data-online)

![Filter URLs outside Google](/learn_images/scraping/extract-urls-webpage/google/step5-filter-2.png#2533x1355)

And here are your external links:

![Google Links results](/learn_images/scraping/extract-urls-webpage/google/step6-results.png#2371x1432)

## Example with Amazon Results

This is a step-by-step example with the Amazon products page. We want to extract all links from this page.

![Amazon products listing](/learn_images/scraping/extract-urls-webpage/amazon/amazon-speakers-results.png#1926x1588)

Open the Chrome Developer Tools:

![Open Chrome Developer Tools](/learn_images/scraping/extract-urls-webpage/amazon/amazon-open-developer-tools.png#1930x1650)

Copy-paste the JavaScript code:

![Copy-Paste code](/learn_images/scraping/extract-urls-webpage/amazon/amazon-copy-paste-code.png#2162x1830)

And run it by pressing `Enter`:

![Run code](/learn_images/scraping/extract-urls-webpage/amazon/amazon-run-code.png#2162x1830)

With Datablist [CSV editor](/csv-editor), create a collection.

![Create collection](/learn_images/scraping/extract-urls-webpage/amazon/amazon-create-collection.png#2185x1098)

Paste your scrapped URLs:

![Paste scrapped URLs](/learn_images/scraping/extract-urls-webpage/amazon/import-from-pasting.png#2195x1202)

Your scrapped URLs are listed on the tool. Filter the links, [deduplicate the results](/features/duplicates-remover), and [clean your scraped data](/how-to/clean-web-scraping-data-online).

![Scrapped Amazon URLs](/learn_images/scraping/extract-urls-webpage/amazon/amazon-results.png#2643x1366)








Extracy all URLs from a webpage

Here is a quick JavaScript snippet to extract all URLs from a webpage fast with Google Chrome Developer Tools. No Browser Extension is required!
The JavaScript code generates a list of URLs in CSV format with the anchor texts, and a boolean to know if the URL is internal or external to the current website. Just copy-paste the results into Google Sheets or with the Datablist CSV editor.
Notes: The code works also with Firefox, Safari, etc.

Step 1: Run JavaScript code in Google Chrome Developer Tools
Open Google Chrome Developer Tools with Cmd + Opt + i (Mac) or F12 (Windows). And click on the Console tab.
Copy-Paste the following JavaScript code and press Enter.

const results = [
    [&#39Url&#39, &#39Anchor Text&#39, &#39External&#39]
];
var urls = document.getElementsByTagName(&#39a&#39);
for (urlIndex in urls) {
    const url = urls[urlIndex]
    const externalLink = url.host !== window.location.host
    if(url.href &amp&amp url.href.indexOf(&#39://&#39)!==-1) results.push([url.href, url.text, externalLink]) // url.rel
}
const csvContent = results.map((line)=&gt{
    return line.map((cell)=&gt{
        if(typeof(cell)===&#39boolean&#39) return cell ? &#39TRUE&#39: &#39FALSE&#39
        if(!cell) return &#39&#39
        let value = cell.replace(/[\f\n\v]*\n\s*/g, &quot\n&quot).replace(/[\t\f ]+/g, &#39 &#39);
        value = value.replace(/\t/g, &#39 &#39).trim();
        return `&quot${value}&quot`
    }).join(&#39\t&#39)
}).join(&quot\n&quot);
console.log(csvContent)
Open Google Developer Tools
Step 2: Copy-paste exported URLs into a CSV file or spreadsheet tools
The JavaScript code will extract all URLs from the webpage with the following values:
URL - The link URL
Anchor Text - The label associated with the link. Called &ampquot;Anchor Text&ampquot;.
External - A boolean (TRUE, FALSE)

The result is printed on the console using CSV format. You can copy-paste the result into spreadsheet tools or with our online CSV editor. Or create a new text file with your text editor and save the file with a .csv extension.
Google automatically adds a Copy button when you get a lot of results.
Open Google Results
Step 3: Filter CSV data to get relevant links
To filter quickly your links, open Datablist CSV editor. Create a collection and paste your CSV data. Check the Google and Amazon examples below for a more step-by-step guide.
Create new collection
Select &amp#39;First row contains header&amp#39; and click on the &ampquot;+&ampquot; button for each column to import them.
Paste results
Then you will be able to use text search or filter on a specific property.
You can use Datablist to consolidate all your CSV data. And Datablist comes with an algorithm for deduplication to have a clean list or URLs.
Example with Google Results
This is a step-by-step example with the Google results page. We want to extract all external links from a Google search result.
Step 1: Search for a Google term that you want to extract links.
A nice tip is to add the &ampamp;num=100 parameter to the URL to force Google into showing 100 results per page.
Force 100 results
Open Google Results
Step 2: Copy-paste the code and run it
Open the console in Google Developer Tools, paste the JavaScript code and press Enter.
Copy-paste code
Copy the results. Select the line and ctrl + c or use the Copy button when Chrome shows it.
Run JavaScript code
Step 3: Filter results
Open Datablist CSV editor, create a collection, and paste the copied URLs.
Note: Don&amp#39;t forget to select the option First row contains headers and to click the &ampquot;+&ampquot; button as shown.

Copy-Paste results in Datablist
Once imported. Click &ampquot;Filter on property&ampquot; in the &ampquot;External&ampquot; column.
Create Filters
And filter items with External to TRUE and Url that does not contain google. Check our guide on how to clean your scraped data
Filter URLs outside Google
And here are your external links:
Google Links results
Example with Amazon Results
This is a step-by-step example with the Amazon products page. We want to extract all links from this page.
Amazon products listing
Open the Chrome Developer Tools:
Open Chrome Developer Tools
Copy-paste the JavaScript code:
Copy-Paste code
And run it by pressing Enter:
Run code
With Datablist CSV editor, create a collection.
Create collection
Paste your scrapped URLs:
Paste scrapped URLs
Your scrapped URLs are listed on the tool. Filter the links, deduplicate the results, and clean your scraped data.
Scrapped Amazon URLs


An Impressum is a legally mandated statement in German-speaking countries, required in publications like books, newspapers, websites, and business correspondence. It must include details about the ownership and authorship of the document.
Most websites in German-speaking countries have an Impressum page with their contact information.
Scraping this data is perfect for lead generation. Impressum pages contain valuable data such as:
<ul>
<li>Founder/CEO Name</li>
<li>Job title</li>
<li>Phone Number</li>
<li>Email Address</li>
<li>Postal Address</li>
</ul>
The information is available but not in a structured format. Until now, it was hard to extract automatically that information. But with AI, I'll show you how to extract those contact information in a structured format.
In this guide, I show you how to use Datablist AI Agent to scrape a list of websites and extract any information you want from the Impressum pages.
<h2 id="user-content-step-1-import-your-list-of-websites"><a href="#step-1-import-your-list-of-websites" aria-hidden="true" tabindex="-1"></a>Step 1: Import your list of websites</h2>
First, import a CSV/Excel file with a list of websites you want to scrape. Datablist is a lead management tool with superpowers. One of them is the Datablist AI Agent. Our AI Agent understands text and can scrape website to find relevant data.
To import your CSV file, create an empty collection and click import. Or click on the "Start with a CSV/Excel file" button from the home screen.
<img src="/learn_images/scraping/scrape-impressum/create-collection-from-file.png#2270x1361" alt="Import CSV File">
This is my file imported. It contains two columns: the name of the company, and its website.
<img src="/learn_images/scraping/scrape-impressum/list-websites-blur.png#2423x1327" alt="List of German websites to scrape">
<h2 id="user-content-step-2-write-a-prompt-that-works"><a href="#step-2-write-a-prompt-that-works" aria-hidden="true" tabindex="-1"></a>Step 2: Write a prompt that works</h2>
Now, we will ask an AI Agent to visit each website, find the Impressum page link, and then read the Impressum page to extract contact information.
Click on the "Enrich" menu and select "AI Agent".
<img src="/learn_images/scraping/scrape-impressum/ai-agent-enrichment-blur.png#2297x1229" alt="Open AI Agent">
We must write a prompt with the website URL to visit, and the data we want to extract. And we can add some tips to find the Impressum page.
Here is my prompt:
<pre><code>Visit {{WEBSITE}} and scrape the impressum page to extract the following information: 
- Founder name
- Job Title
- Email Address
- Phone Number
The Impressum page is usually on the URL /impressum
</code></pre>
You can ask for more (or less) data points. The website URL is defined as a placeholder. And Datablist will run a personalized prompt for each line.
To use a variable for the website, enter 2 brackets characters and select the Website property from your collection.
<img src="/learn_images/scraping/scrape-impressum/write-prompt.png#1526x930" alt="Write a prompt">
<h2 id="user-content-step-3-configure-expected-outputs"><a href="#step-3-configure-expected-outputs" aria-hidden="true" tabindex="-1"></a>Step 3: Configure expected outputs</h2>
After the prompt, we need to configure the expected outputs. The Agent uses the output's name and description along with the prompt to understand the mission.
Here, we have:
<ul>
<li>Founder Name - Description: The name of the founder. Empty if not found.</li>
<li>Job Title - Description: The job title of the founder, when available.</li>
<li>Email Address - Description: Email address found on the impressum page.</li>
<li>Phone Number - Description: The company phone number.</li>
</ul>
<img src="/learn_images/scraping/scrape-impressum/expected-outputs.png#1758x1248" alt="Expected outputs">
<h2 id="user-content-step-4-add-outputs-to-the-collection"><a href="#step-4-add-outputs-to-the-collection" aria-hidden="true" tabindex="-1"></a>Step 4: Add outputs to the collection</h2>
Click "Continue to outputs configuration". The expected outputs configured in the previous step appear here.
Select "+" to add a new property (=column) to your collection for each output.
<img src="/learn_images/scraping/scrape-impressum/add-properties.png#2292x1380" alt="Add properties">
You can also create a property to see the error message when the agent is not able to perform the task.
<h2 id="user-content-step-5-run-the-enrichment"><a href="#step-5-run-the-enrichment" aria-hidden="true" tabindex="-1"></a>Step 5: Run the enrichment</h2>
The last step, click on "Instant run" to run the agent.
<img src="/learn_images/scraping/scrape-impressum/results-blur.png#2537x1398" alt="Scraping results">
The "Error Msg" property shows a text when the agent is not able to reach the website or if no Impressum page is found.

An Impressum is a legally mandated statement in German-speaking countries, required in publications like books, newspapers, websites, and business correspondence. It must include details about the ownership and authorship of the document. 

Most websites in German-speaking countries have an Impressum page with their contact information.

**Scraping this data is perfect for lead generation.** Impressum pages contain valuable data such as:

- Founder/CEO Name
- Job title
- Phone Number
- Email Address
- Postal Address

The information is available but not in a structured format. Until now, it was hard to extract automatically that information. **But with AI, I'll show you how to extract those contact information in a structured format.**

In this guide, I show you how to use Datablist AI Agent to scrape a list of websites and extract any information you want from the Impressum pages.

## Step 1: Import your list of websites

First, import a CSV/Excel file with a list of websites you want to scrape. Datablist is a lead management tool with superpowers. One of them is the Datablist AI Agent. Our AI Agent understands text and can scrape website to find relevant data.

<aside class="text-center mb-5">
 <div class="mb-2">
 <a href="https://app.datablist.com/" className="btn btn-primary btn-large" target="_blank">
 Open Datablist
 </a>
 </div>
</aside>

To import your CSV file, create an empty collection and click import. Or click on the "Start with a CSV/Excel file" button from the home screen.

![Import CSV File](/learn_images/scraping/scrape-impressum/create-collection-from-file.png#2270x1361)

This is my file imported. It contains two columns: the name of the company, and its website.

![List of German websites to scrape](/learn_images/scraping/scrape-impressum/list-websites-blur.png#2423x1327)


## Step 2: Write a prompt that works

Now, we will ask an AI Agent to visit each website, find the Impressum page link, and then read the Impressum page to extract contact information.

Click on the "Enrich" menu and select "AI Agent".

![Open AI Agent](/learn_images/scraping/scrape-impressum/ai-agent-enrichment-blur.png#2297x1229)

We must write a prompt with the website URL to visit, and the data we want to extract. And we can add some tips to find the Impressum page.

Here is my prompt:

```
Visit {{WEBSITE}} and scrape the impressum page to extract the following information: 
- Founder name
- Job Title
- Email Address
- Phone Number
The Impressum page is usually on the URL /impressum
```

You can ask for more (or less) data points. The website URL is defined as a placeholder. And Datablist will run a personalized prompt for each line.

**To use a variable for the website, enter 2 brackets characters and select the Website property from your collection.**

![Write a prompt](/learn_images/scraping/scrape-impressum/write-prompt.png#1526x930)

## Step 3: Configure expected outputs

After the prompt, we need to configure the expected outputs. The Agent uses the output's name and description along with the prompt to understand the mission.

Here, we have:

- **Founder Name** - Description: The name of the founder. Empty if not found.
- **Job Title** - Description: The job title of the founder, when available.
- **Email Address** - Description: Email address found on the impressum page.
- **Phone Number** - Description: The company phone number.

![Expected outputs](/learn_images/scraping/scrape-impressum/expected-outputs.png#1758x1248)


## Step 4: Add outputs to the collection

Click "Continue to outputs configuration". The expected outputs configured in the previous step appear here.

Select "+" to add a new property (=column) to your collection for each output.

![Add properties](/learn_images/scraping/scrape-impressum/add-properties.png#2292x1380)

You can also create a property to see the error message when the agent is not able to perform the task.

## Step 5: Run the enrichment

The last step, click on "Instant run" to run the agent.

![Scraping results](/learn_images/scraping/scrape-impressum/results-blur.png#2537x1398)

The "Error Msg" property shows a text when the agent is not able to reach the website or if no Impressum page is found.

Automatically extract contact information on the Impressum page

An Impressum is a legally mandated statement in German-speaking countries, required in publications like books, newspapers, websites, and business correspondence. It must include details about the ownership and authorship of the document. 
Most websites in German-speaking countries have an Impressum page with their contact information.
Scraping this data is perfect for lead generation. Impressum pages contain valuable data such as:
Founder/CEO Name
Job title
Phone Number
Email Address
Postal Address

The information is available but not in a structured format. Until now, it was hard to extract automatically that information. But with AI, I&amp#39;ll show you how to extract those contact information in a structured format.
In this guide, I show you how to use Datablist AI Agent to scrape a list of websites and extract any information you want from the Impressum pages.
Step 1: Import your list of websites
First, import a CSV/Excel file with a list of websites you want to scrape. Datablist is a lead management tool with superpowers. One of them is the Datablist AI Agent. Our AI Agent understands text and can scrape website to find relevant data.
To import your CSV file, create an empty collection and click import. Or click on the &ampquot;Start with a CSV/Excel file&ampquot; button from the home screen.
Import CSV File
This is my file imported. It contains two columns: the name of the company, and its website.
List of German websites to scrape
Step 2: Write a prompt that works
Now, we will ask an AI Agent to visit each website, find the Impressum page link, and then read the Impressum page to extract contact information.
Click on the &ampquot;Enrich&ampquot; menu and select &ampquot;AI Agent&ampquot;.
Open AI Agent
We must write a prompt with the website URL to visit, and the data we want to extract. And we can add some tips to find the Impressum page.
Here is my prompt:
Visit {{WEBSITE}} and scrape the impressum page to extract the following information: 
- Founder name
- Job Title
- Email Address
- Phone Number
The Impressum page is usually on the URL /impressum
You can ask for more (or less) data points. The website URL is defined as a placeholder. And Datablist will run a personalized prompt for each line.
To use a variable for the website, enter 2 brackets characters and select the Website property from your collection.
Write a prompt
Step 3: Configure expected outputs
After the prompt, we need to configure the expected outputs. The Agent uses the output&amp#39;s name and description along with the prompt to understand the mission.
Here, we have:
Founder Name - Description: The name of the founder. Empty if not found.
Job Title - Description: The job title of the founder, when available.
Email Address - Description: Email address found on the impressum page.
Phone Number - Description: The company phone number.

Expected outputs
Step 4: Add outputs to the collection
Click &ampquot;Continue to outputs configuration&ampquot;. The expected outputs configured in the previous step  appear here.
Select &ampquot;+&ampquot; to add a new property (=column) to your collection for each output.
Add properties
You can also create a property to see the error message when the agent is not able to perform the task.
Step 5: Run the enrichment
The last step, click on &ampquot;Instant run&ampquot; to run the agent.
Scraping results
The &ampquot;Error Msg&ampquot; property shows a text when the agent is not able to reach the website or if no Impressum page is found.


Scraping

What are the most common URL paths for "About Us" pages?

How to extract the country code from a list of Shopify stores?

How to extract all URLs from a webpage?

How to scrape Impressum pages and extract data?

Join the future of list management