Data Sources & Scraping Policy

1. What we collect

• Job title, department, location, work mode, employment type, seniority.
• Job description and requirements as published on the employer's page.
• Salary range where the employer publishes it.
• Apply URL that points back to the employer's own system.
• Company name, overview, industry, size, HQ location, public logo, and public social links.

2. Where the data comes from

• Employer career pages we discover via their homepage, sitemap, or robots.txt.
• Structured feeds employers publish deliberately (JSON-LD JobPosting, RSS/Atom, standard ATS endpoints).
• User submissions of a public job URL — always re-verified before ingestion.

We do not scrape LinkedIn, Indeed, or other aggregator sites. Every job in Talenthic ties back to a first-party employer URL.

3. How we crawl

• We identify ourselves with a descriptive User-Agent: TalenthicBot/1.0 (+https://talenthic.com/data-sources).
• We respect robots.txt and X-Robots-Tag, including noindex, noarchive, and per-user-agent disallow rules.
• We rate-limit to no more than a small handful of parallel requests per host, back off aggressively on 429 / 5xx, and re-crawl on a rolling schedule (typically hourly per host at most, daily on average).
• We cache the last successful content hash and skip re-processing if unchanged.
• We do not bypass authentication, CAPTCHA, paywalls, or rate-limit tokens. We do not sign in to third-party sites.

4. Enrichment with AI

After ingestion, ML models extract structured fields (skills, seniority, salary), generate short summaries, and detect near-duplicates. Extracted values are cross-checked against the source text; anything the model invents that is not supported by the source is discarded. See the Privacy Policy for the AI subprocessors involved.

5. Employer verification

Employers can claim their profile by emailing us from a corporate domain address. A verified badge appears once we confirm ownership. Verified employers can edit their overview, industry, logo, and social links, and can suppress specific listings.

6. Opt-out

Employers who do not want Talenthic to index them have three options:

• Robots.txt — disallow TalenthicBot for the paths you want blocked. Effective on the next crawl cycle.
• Meta / header — serve <meta name="TalenthicBot" content="noindex"> or an X-Robots-Tag: talenthicbot: noindex header on the pages you do not want stored.
• Email removal — email inknamestudio@gmail.com from a corporate domain. We remove and add the domain to our opt-out list, usually within one business day.

7. Freshness & expiry

• We re-check each job on a rolling schedule and deactivate any listing not seen for 14 days.
• Deactivated jobs are removed from search but retained for 90 days for historical analytics; then purged.
• Deleted-by-employer jobs are hard-deleted within 24 hours.

8. What we never do

• Solicit candidate résumés or applications on the employer's behalf.
• Sell scraped data to third parties as raw dumps.
• Train a general-purpose foundation model on scraped content.
• Bypass any technical access control.

9. Questions

Opt-outs, corrections, partnerships: inknamestudio@gmail.com.