| Browser CSV analysis |
The browser on this device |
CSV rows and cells, analysis choices, results, and an optional local session. |
The released workbench has no CSV upload endpoint. It does not collect public sources or offer hosted AI. |
| Website delivery |
Website host and network providers |
Standard request and connection metadata needed to deliver and protect a page. |
The released pages contain no advertising or analytics scripts. Provider processing terms need review. |
| Windows collection |
The public site, search service, or provider API selected by the researcher |
Query terms, target URLs, request metadata, and a researcher-owned credential when that provider path uses one. |
Recipients and exact fields depend on the selected source and provider. Source terms remain the researcher's responsibility. Importing a caption, transcript or media file sends nothing. Where transcription is available, it runs on this computer, and the app downloads no model; the installed build does not include transcription in this release. Section 7 of the test guide names the hosts that the researcher-run model download contacts. |
| Find sources and keyless web search |
api.gdeltproject.org (GDELT), api.wikimedia.org (Wikimedia) and api.openalex.org (OpenAlex), or first a SearXNG instance that you set. api.github.com, hn.algolia.com and gamma-api.polymarket.com only when the researcher adds GitHub, Hacker News or Polymarket to the plan. |
The approved plan's search words, sent only after Approve and find sources, and the Collabysis User-Agent. GDELT, Wikimedia and OpenAlex receive the same search words only. No account, key or email address is sent. |
Keyless Web search sends the question keywords to api.gdeltproject.org (GDELT), api.wikimedia.org (Wikimedia) and api.openalex.org (OpenAlex). When you set a SearXNG instance, it is asked first, and the three hosts receive the keywords only when it returns nothing. A source you skip in the plan receives nothing. Collabysis sends no request to DuckDuckGo. None of these hosts is an AI service. Without a web search key, a plan names no Web research, Find sources sends no Web search request, and the Web search card is held for the run. A web search key counts when it is saved in Connect your sources or set in the environment that starts Collabysis. Web pages topic discovery uses the same keyless search, and so does Web research when a web search key is saved and the keyless backend is chosen. In each Find sources, the app reads the robots.txt of each of these hosts before its first request to that host. A host receives no search request when its robots.txt disallows the request, when it refuses the robots.txt request (HTTP 401 or 403), when it asks for a slower pace (HTTP 429), when the robots.txt request fails with a server error, or when the host cannot be reached. The Sources step then names the source with the reason. A host with no robots.txt (HTTP 404, 410 or another 4xx answer) is searched. The app honors Retry-After and refusals. With a paid or trial subscription license, the auto Web search backend and no web search key, a plan also names Web research, and its searches go through collabysis.com; Included searches and AI describes those requests. |
| Start collection from a question's plan |
For Web research, which a plan names only when a web search key is saved, the search service the researcher chooses or, with the keyless backend, the hosts of the Find sources row, then the website of each page its results link to, such as doi.org, openalex.org or en.wikipedia.org. With Follow relevant page links checked, as it is by default, Web research also reads pages those pages link to on the same websites and their subdomains, including next pages and feeds. With it unchecked, Web research reads only the result pages. These hosts depend on the results and are not known in advance. For Public data, when the researcher keeps it on the Sources step, api.us.socrata.com for portal search (data.gov search through catalog-old.data.gov is held in this release and sends no request), then the host of the dataset the researcher chooses or, with none chosen, of the portal search's top match, such as a city data portal; or the host of a faster connector the researcher chooses (earthquake.usgs.gov; geocoding-api.open-meteo.com, archive-api.open-meteo.com and api.open-meteo.com; services3.arcgis.com). Public data portal search also contacts www.socrata.com, the host to which api.us.socrata.com redirects its robots.txt request. For the Hacker News source, hacker-news.firebaseio.com for the items of the discussion the researcher names. For each keyless source the plan keeps: api.bsky.app (Bluesky), mastodon.social (Mastodon) and api.wikimedia.org (Wikimedia) in both plans, and api.openalex.org (OpenAlex) and api.crossref.org (Crossref) in a research plan. api.gdeltproject.org (GDELT) when the researcher adds GDELT to the plan, and on each Web research search that uses the keyless backend, unless the researcher's SearXNG instance answers. With a YouTube Data API key saved, the plan also lists YouTube, which sends the search words and your key to www.googleapis.com (YouTube Data API) and reads no robots.txt. With a paid or trial subscription license, the auto Web search backend and no web search key, a plan also names Web research, and its searches go to collabysis.com, which forwards them to api.search.brave.com; Included searches and AI describes those requests. |
The search hosts receive two kinds of search. Each keyless source card sends the approved plan's first content phrase, at most four words, to its host: api.bsky.app, api.wikimedia.org, api.openalex.org and api.crossref.org, and api.gdeltproject.org when GDELT is in the plan. The Mastodon card sends mastodon.social up to two tags derived from that phrase, and each tag is read as its own tag timeline; with replies selected, the two tags are sent together as one post text search. A keyless Web research search sends up to eight keywords of the question to api.gdeltproject.org, api.wikimedia.org and api.openalex.org. Every search host receives the Collabysis User-Agent. A result page, a linked page or a dataset host receives a request for that page or dataset with the Collabysis User-Agent; the app adds no question text to that request. No keyless source receives an account, key or email address. A saved YouTube key goes only to www.googleapis.com. |
None of these hosts is an AI service. A skipped source is not contacted. Web research reads at most the number of pages set in Pages to extract (20 by default). Unchecking Extract the pages found by web search stops all page reading. Each keyless source reads its host's robots.txt before its first request. Web research reads the robots.txt of each search host before it searches it, and the robots.txt of each website before the first page it reads there. One Start can read the same host's robots.txt more than once. When GitHub, Hacker News, Polymarket or a Public data host redirects its robots.txt request, the app follows the redirect at most five times and sends the target host only the redirected robots.txt request, with no question text. The app honors Retry-After and refusals, and paces GDELT, Wikimedia, OpenAlex and Crossref per host across every run on the computer. Let Collabysis AI choose searches and pages (beta), a switch for Web research on the Sources step, is off by default. With it on, Collabysis AI proposes the searches and chooses the result pages to read inside the approved plan, and the app stops the run at the budget of searches, pages and AI calls shown before Start: at most 15 searches, the number set in Pages to extract and 20 AI calls. With the switch on, each search goes to the keyless hosts of the Find sources row, whatever web search key or license is set. Collabysis AI receives the question, the problem statement, the construct labels and the offered source names and, to choose pages, the titles, snippets and hosts of up to 20 search results or linked pages per choice, each title cut to 200 characters and each snippet to 500; it never receives a page body, a dataset row or a key. With the switch on, the pages read depend on the search results and are not known in advance. The same robots.txt, Retry-After and refusal rules apply to every search and page. AI routes and hosts names the hosts that receive the AI requests. |
| Wikipedia depth |
wikimedia.org (daily pageviews) and api.wikimedia.org (revision lists); en.wikipedia.org for its robots.txt only. |
Each chosen article's title and the Collabysis User-Agent. No account, key or email address is sent. |
For Wikipedia depth, the app sends each chosen article's title to wikimedia.org (daily pageviews) and to api.wikimedia.org (revision lists), after requesting each host's robots.txt. Both hosts redirect that robots.txt request; the app does not follow those redirects and contacts no other host for them. Before the first article it also reads the robots.txt of en.wikipedia.org. The app never requests the /w/ or /api/ paths that en.wikipedia.org's robots.txt disallows, skips any article whose Wikipedia address that file disallows, and the robots.txt setting does not change this. |
| Download media |
The host of each media file in the collected records. These hosts are not known in advance. |
A request for the media file and its host's robots.txt, with the user agent Collabysis-Media/1.0. No question text, account, key or email address is sent. |
Download media requests each media file's host after reading robots.txt at most once per origin per run (scheme, host and port), one request per second per host, honors Retry-After, and sends a host nothing more after a refusal. These hosts come from the collected records and are not known in advance; YouTube pages and videos are never requested. |
| Publisher caption |
The host of the caption URL the researcher enters. These hosts are not known in advance. |
A request for the caption file and its host's robots.txt, with the user agent Collabysis/3 publisher-caption. No question text, account, key or email address is sent. |
Applying a publisher caption reads the robots.txt of the caption URL's host, then requests the caption file from that host; YouTube hosts are refused, and the robots.txt setting does not change this. |
| robots.txt decisions, declaration and export |
This computer |
The robots.txt decision per host, the first-run declaration and the pseudonym key. |
Each collection receipt records, per host, the robots.txt answer the app saw, the rule it matched and the decision it took. Hosts whose robots.txt was not requested before the run stopped are stated as not checked. For X, YouTube (official API), Instagram and TikTok, which use a provider API, no robots.txt is read, and the export says so. With a Saudi Arabia or United States declaration, a Disallow for all crawlers (User-agent: *) does not stop a request by default. You can turn robots.txt honouring on for each source or site in Settings. With an EU or EEA declaration, an Other declaration or no declaration, Collabysis honours robots.txt, and Settings offers no robots.txt setting. The robots.txt setting covers only Hacker News, the open data sources, data file links, the keyless research searches other than Wikipedia, Web pages, the pages that Web research extracts and Download media. It never applies when a robots.txt group names Collabysis or when a host refuses or rate-limits the request, and it has no effect on Find sources, Mastodon, Bluesky or Wikipedia depth. The robots.txt setting does not apply to Wikipedia or the other Wikimedia sites; Collabysis always honours their robots.txt. A change to a source's robots.txt setting applies from the next request. A run records the declaration it started with; if you save the declaration again during a run, robots.txt is honoured for the rest of that run. Web pages requests to a site where robots.txt is not honoured identify Collabysis as "Collabysis-Research/1.0 (+public-data collection)", without the clause "respects robots.txt". A Disallow in a robots.txt group that names Collabysis always stops the request it covers. A refused, rate-limited, unreadable or unreachable robots.txt request also stops the request. Apart from Wikipedia pages, Wikipedia depth and the own-key routes, a 429 answer ends collection from that site for the rest of the run, and its Retry-After is kept as the site's wait for later runs. Apart from Wikipedia pages, Wikipedia depth and the own-key routes, a 403 that carries a valid Retry-After, or a rate-limit reset that the source reads, is handled as a 429 unless the source also reads a CAPTCHA signal in it; any other 403, and any answer other than a 429 in which the source reads a CAPTCHA signal in the headers, cookies or JSON fields, ends collection from that site until you clear it under Stopped sites in Settings. Wikipedia pages and Wikipedia depth keep their own handling: a 429's Retry-After is kept as the site's wait for later runs, a 429 that asks for at most 60 seconds (Wikipedia pages) or 120 seconds (Wikipedia depth) is waited out once and the request is sent again, any other 429 ends the run, a 403 ends the run, and a 403 or a CAPTCHA answer stores no site stop. A site you enter under Stopped sites receives no request, robots.txt included, until you remove it. A declaration change does not remove it. Each collection receipt records every stop decision with the source that received it. Requests to X, the YouTube Data API and ScrapeCreators use your own key and read no robots.txt. A 403, 429 or CAPTCHA answer on these routes stores no site stop, because such an answer can concern your key or quota. A site you enter under Stopped sites receives no request from these routes either. These routes contact only the host of the provider whose key you save: api.twitter.com (X), www.googleapis.com (YouTube Data API) and api.scrapecreators.com (ScrapeCreators). A paid Web search backend whose key you save receives Web research queries in place of the keyless search, at api.search.brave.com, api.exa.ai, google.serper.dev or api.parallel.ai. On the first run the app asks where you work and your role; the answer is stored on this computer, in each collection receipt and in the method note of a coded export, and is sent nowhere. Export and share default to a copy in which author names and handles are replaced by pseudonyms from a key stored apart from the data on this computer; you can destroy the key. |
| Start collection from Web pages |
For Web pages, the website of each starting page. A starting page is a URL you paste under URLs I paste or a page that Topic discovery finds with the keyless search of the Find sources row. These websites depend on your URLs and the search results and are not known in advance. With Follow relevant page links checked, as it is by default, Web pages also reads the pages that those pages link to. With Follow in-scope pagination checked, as it is by default, Web pages also reads next pages. With Collect in-scope feed links checked, which is off by default, Web pages also reads feeds. With Record sitemap locations as proposals checked, which is off by default, Web pages also reads the robots.txt Sitemap lines of each website it reads a page from, including subdomains, and requests that website's /sitemap.xml only when robots.txt names no sitemap. |
A starting page, linked page, next page or feed receives a request for that page with the Collabysis User-Agent. The app adds no question text to that request. A sitemap request asks for the sitemap file with the same User-Agent. No account, key or email address is sent. |
None of these websites is an AI service. Linked pages, next pages and feeds stay on the starting websites and their subdomains. Uncheck Follow relevant page links, Follow in-scope pagination, Collect in-scope feed links or Record sitemap locations as proposals to stop that request. Choose Topic discovery under Starting pages to request no pasted URL. Maximum pages in this collection and Maximum page requests limit the total. The pages that a sitemap lists are not requested. |
| Optional Windows AI assist |
The provider selected by the researcher |
Route-specific context. Ask and AI-authored analysis receive the question, schema, and method list; AI-authored analysis may also receive prior method names. With provider opt-in, method selection receives the researcher's methodology text, schema names and roles, fixed method catalog, and explicitly selected design-context fields. After disclosure, Define research variables receives selected source-text segments plus their row IDs and content-state markers, the researcher's field definitions, and a source-snapshot digest. Scoped PDFs also include selected-page scope metadata. Semantic Scout receives bounded excerpts after BYO-key opt-in. |
Ask, method selection, and AI-authored analysis do not send dataset rows, cell values, or the full CSV. Define research variables sends selected record context; it does not send unselected rows or the full CSV. Semantic Scout removes record identifiers, URLs, source labels, and other record metadata before its request. Provider retention terms require separate review. |
| Downloads |
An external release host |
Standard request and connection metadata for an installer or ZIP. |
The website redirects artifact downloads to that host. Check the version and SHA-256 value before running a file. |