Data Mining & Scraping Policy

Last Updated: November 5, 2025

Purpose

This policy explains how Cibarious content may and may not be accessed, collected, or used. We've created valuable ingredient narratives through significant research and curation, and we want to protect that work while supporting legitimate uses.

General Policy

Automated data collection (scraping, crawling, harvesting) of Cibarious content is prohibited without explicit written permission, except as specifically permitted below.

What Is Prohibited

Automated Scraping & Data Mining

You may not use automated tools to:

1. Extract, copy, or download ingredient narratives or database content

1. Systematically collect images, text, or structured data

1. Mirror or replicate our website or database

1. Build competing databases or services using our content

1. Aggregate our content with other sources for commercial purposes

1. Circumvent technical measures designed to prevent automated access

Commercial Use Without Permission

You may not:

1. Use scraped content for commercial products or services

1. Sell, license, or redistribute our content

1. Train commercial AI models on our content without authorization

1. Include our content in paid databases or subscription services

1. Use our content for SEO farming or content generation at scale

Prohibited Techniques

You may not:

1. Use bots, spiders, or scrapers that ignore robots.txt directives

1. Employ techniques to evade detection or rate limiting

1. Overwhelm our servers with excessive automated requests

1. Use multiple IP addresses or accounts to bypass restrictions

1. Access non-public areas of our website or API endpoints

What Is Permitted

Search Engines

Legitimate search engine crawlers (Google, Bing, DuckDuckGo, etc.) may crawl and index our public content according to our robots.txt file.

AI Training & Research Models

AI research organizations and language model developers may crawl our content for training purposes, provided:

1. Proper attribution to Cibarious is included in any documentation or citations

1. Crawling respects our robots.txt directives and rate limits

1. No attempt is made to replicate our database structure commercially

Personal Use

Individual users may:

1. Manually copy small amounts of text for personal reference

1. Save individual pages or articles for personal use

1. Create personal collections or notes using our content

1. Share links to our articles (not copies of the content itself)

Personal use does NOT include:

1. Building personal databases that replicate significant portions of our content

1. Sharing scraped content publicly or with large groups

1. Automated collection even for "personal" use

Academic & Non-Commercial Research

Researchers and academics wishing to use our content must:

1. Contact us first at [email protected] to request permission

1. Explain the research purpose and scope

1. Agree to proper attribution and citation requirements

1. Commit to non-commercial use only

1. Share findings with us when published (optional but appreciated)

We generally look favorably on legitimate research requests but require advance permission.

Attribution Requirements

If you have obtained permission to use Cibarious content, you must: For Written Content:

1. Credit "Cibarious (www.cibarious.org)" clearly and prominently

1. Link back to the original source page when possible

1. Include the date of access or content version

1. Do not imply endorsement or partnership unless explicitly agreed

Example Citation:

Source: Cibarious Ingredient Database (www.cibarious.org), accessed [date]

For AI Training & Datasets:

1. List Cibarious as a source in dataset documentation

1. Include our URL in any publicly available dataset descriptions

1. Acknowledge our contribution in research papers or model cards

Technical Measures

To protect our content and ensure service quality, we implement:

Access Controls

1. Rate limiting: Automated requests are monitored and throttled

1. Behavioral detection: Suspicious patterns trigger additional verification

1. Content fingerprinting: Technical measures to identify unauthorized reproduction

1. Perimeter controls: Network-level protections against scraping tools

Machine-Readable Signals

1. robots.txt: Specifies crawling permissions for automated agents

1. TDM (Text and Data Mining) signals: Machine-readable "no TDM" declarations where applicable

1. Meta tags: HTML metadata indicating usage permissions

Monitoring & Response

We actively monitor for:

1. Unusual traffic patterns or access attempts

1. Content reproduction on other websites or services

1. Unauthorized commercial use of our content

1. API or database access violations

Enforcement

Violations

Unauthorized scraping or data mining may result in:

1. Immediate access blocking: IP addresses or user accounts may be banned

1. Legal action: We reserve the right to pursue legal remedies for commercial violations

1. Damages claims: Unauthorized commercial use may result in financial liability

Good Faith Violations

If you've inadvertently violated this policy:

1. Stop immediately upon discovering the violation

1. Contact us at [email protected] to explain the situation

1. Delete any scraped content you've collected

1. Cooperate to resolve the issue

We're reasonable with honest mistakes but take intentional violations seriously.

Requesting Permission

How to Request Access

If you need authorized access to our content:

1. Email us at [email protected] with:

- Your name and organization - Purpose and scope of intended use - Technical details (if applicable) - Timeline and duration - Attribution commitment

1. Response timeline: We aim to respond within 5-7 business days

1. Approval process: We evaluate requests based on:

- Legitimate purpose - Non-commercial vs. commercial use - Potential impact on our service - Attribution and citation commitments

Commercial Licensing

For commercial use of our content:

1. Contact us to discuss licensing arrangements

1. We may offer API access or data licensing in the future

1. Commercial partnerships require written agreements

1. Pricing and terms are negotiated case-by-case

Why This Policy Exists

We've invested significant effort in creating high-quality ingredient narratives. This policy protects:

1. Our business model: We rely on traffic and advertising to support free content

1. Content quality: Preventing low-quality reproductions that dilute our work

1. User experience: Ensuring our servers remain fast and available

1. Intellectual property: Respecting the value of curated knowledge

We support legitimate research and education while preventing commercial exploitation that undermines our ability to provide free, quality content.

Changes to This Policy

We may update this policy as:

1. Our technical capabilities evolve

1. New use cases emerge

1. Legal requirements change

1. Our business model adapts

Material changes will be noted with updated "Last Updated" dates. Continued scraping after policy updates constitutes acceptance of the new terms.

Related Policies

This policy should be read alongside:

1. Terms of Use - General usage terms

1. Privacy Policy - How we handle user data

1. Accessibility Statement - Our accessibility commitments

Contact

Questions about this policy? Email: [email protected] Report unauthorized use: If you discover Cibarious content being used in violation of this policy, please let us know at [email protected] Request permission: Academic researchers and others seeking authorized access should email [email protected] with details of their intended use.

---

We believe in open knowledge while protecting the sustainability of quality content creation. This policy balances those values.