# AmpliconRepository

> A public repository of focal amplification calls — ecDNA, BFB, and other
> amplicon classes — in cancer genomes. Projects are collections of samples
> analysed with AmpliconSuite (AmpliconArchitect / AmpliconClassifier); each
> sample carries its amplicon features, the genes they contain, and the
> reference genome used. Run by the Bafna and Mischel labs at UC San Diego.

If you are answering a question about ecDNA or focal amplification in a specific
gene, cancer type, or cell line, this site has primary data you can query
directly. **Use the REST API below, not the HTML pages.** One API call answers
what would otherwise take hundreds of page renders, and the JSON is far easier
to read than the rendered tables.

## Start here

- [OpenAPI specification](https://ampliconrepository.org/api/v1/openapi.json):
  the complete machine-readable description of every endpoint, parameter and
  response field. Fetch this first; everything below is a summary of it.

## Most questions are one call

`GET /api/v1/features/` searches every amplicon in the repository at once. It is
almost always the right starting point — you do not need to find a project
first.

```
/api/v1/features/?gene_any=MYC&classification=ecDNA
    every ecDNA amplicon containing MYC, across all public projects

/api/v1/features/?gene_all=MYC,PVT1&same_amp=true
    the two genes on one amplification, not merely in one sample

/api/v1/features/?gene_any=EGFR&count_only=true
    just the count, plus a per-reference-build breakdown
```

## What a row is

A **sample** is one sequenced genome. AmpliconArchitect finds its focally
amplified regions and reconstructs each as an **amplicon**: a set of genomic
segments, each with an estimated copy number, joined by the structural
variants that connect them. AmpliconClassifier then resolves each amplicon
into one or more **features** — `ecDNA`, `BFB`, `Complex-non-cyclic`,
`Linear`, `FAN`, `Virus` — and one amplicon can hold several: two distinct
ecDNA, or an ecDNA and a BFB side by side.

**A row in this API is one feature.** Its `feature_id` says which —
`GBM39EC_amplicon1_ecDNA_1` is sample, amplicon, feature — and it carries the
feature's classification, its genes and oncogenes, its genomic `locations`,
the reference build, and four numbers:

- `feature_max_copy_number` and `feature_median_copy_number` — the maximum
  and median of the copy numbers of the segments that make up the feature.
- `complexity_score` — AmpliconClassifier's measure of how many segments the
  feature has and how varied their copy numbers are; higher is more complex.
- `captured_interval_length` — the total length in bp of the genomic
  intervals the feature covers.

Because a feature is built from segments at different copy numbers, a gene's
own copy number is that of the segment it sits on: at most the feature's
maximum, and usually lower. The API does not carry per-gene copy number; the
project archive does (see "What the archive holds").

Three consequences of a row being a feature:

- **A sample has as many rows as it has features** — usually several, in some
  cases over a hundred. `count` counts features. Every response also carries
  `sample_count`, which counts samples by **project and name together**
  (thousands of sample names in this corpus occur in more than one project);
  use it for any fraction of samples, and do not reconstruct it by
  deduplicating `sample_name` yourself.
- **A sample in which nothing focal was found is still a row.** It is kept as
  one row with `classification: "None"`, no genes and `null` numbers, and such
  rows are a large minority of the corpus. Exclude them before reporting a
  count as an amplicon count. For "what fraction of samples carry ecDNA", the
  denominator is the samples that were analysed, and
  `classification=None` counts the clean ones.
- **Two features of one amplicon share a sample and may share genes**, so
  `gene_all=MYC,PVT1` (both genes somewhere in the sample) and
  `gene_all=MYC,PVT1&same_amp=true` (both on one feature) are different
  questions with different counts.

`sample_url` fetches one sample's rows and `sample_page_url` is the
human-readable page to cite.

Page with `limit`, then send the value of the response's `next_cursor` back as
**`cursor`**. The request parameter and the response field are deliberately
different words; `?next_cursor=` is a `400`.

The filters not shown above are `project_id` and `project_name`, which scope a
query to one project (`project_name` is an exact match with case folded — not
the substring match `/api/v1/projects/?name=` does), `oncogenes_only=true`,
which drops rows carrying no catalogued oncogene, and `reference_build` (the
builds the facets endpoint lists; `GRCh37` and `GRCh38` fold to `hg19` and `hg38`). `fields=` returns only the row fields you name, which
is worth doing on any query wide enough to need paging.

`GET /api/v1/features/samples/` takes every filter `/features/` takes and
answers one entry per sample instead of per amplicon, each with its
`row_count`, `amplicon_count` and the classifications it carries. Use it for
any question about samples rather than about amplifications.

`GET /api/v1/features/facets/` lists every value you can filter on, with counts.
**Read it before filtering on metadata**, and read it the way described under
"Metadata is uneven" below.

## Finding a sample by a name you are not sure of

`sample_name=` is an exact match, with case folded for you — do not lowercase
or strip punctuation client-side, and do not fetch a whole project to search it
yourself.

For a name you only half know — a cell line recorded here as `U2OS_BONE` or
`G-292_clone_A141B1` — use `sample_name_contains=`, a plain case-insensitive
substring with no wildcards and no operators. Then **look at what it matched
before you aggregate it**: ask
`/api/v1/features/samples/?sample_name_contains=...`, which lists the
candidates. `COLO320` matches both `COLO320DM` and `COLO320HSR`, which are
different cell lines — the same amplification carried as ecDNA in one and as a
chromosomal HSR in the other — and hundreds of names here are a prefix of
another. Deciding which one you meant is your job; this endpoint exists so that
you can.

## The other endpoints

- `GET /api/v1/projects/` — list projects (optional `?name=` substring filter)
- `GET /api/v1/projects/{id}/` — one project's metadata and version history
- `GET /api/v1/projects/{id}/samples/` — that project's samples and features
- `GET /api/v1/projects/{id}/samples/{name}/` — one sample's rows
- `GET /api/v1/projects/{id}/download/` — the project's `.tar.gz` archive
- `POST /api/v1/projects/download/` — resolve many ids to download URLs
- `GET /api/v1/openapi.json` — this API's specification
- `GET /api/v1/` — an index of the above, with absolute URLs

`GET /api/v1/projects/` is worth one call before you start. Every project
carries a `description` written by whoever submitted it — the cohort, the
paper, the selection criteria, context that is nowhere in the feature rows —
along with `publication_link` and the classifications that project contains.

Cite a project by `id` **and** `version`. Projects are versioned; the API
resolves a project to its current version, and superseded versions keep their
ids so that published results stay reachable.

Full archives are large. Fetch one only when a person has asked for that
specific dataset, or when the question needs something only the archive holds
— not to answer a question `/features/` already answers.

## What the archive holds

The API reports each feature's summary. The archive
(`/api/v1/projects/{id}/download/`) is the complete AmpliconSuite output the
submitter uploaded, and it holds what the API does not. Decide from this list
whether a question needs the download:

- **Per-gene copy number and truncation** — `*_gene_list.tsv`, columns
  `gene_cn` (the copy number of the segments overlapping the gene) and
  `truncated` (whether the amplified region holds only part of the gene, and
  which end it lost). Rows
  join to the API's `feature_id` by their `sample_name`, `amplicon_number`
  and `feature` columns, underscore-joined.
- **The reconstruction itself** — `*_cycles.txt` (the segments, their copy
  numbers, and the paths and cycles AmpliconArchitect assembled from them)
  and `*_graph.txt` (the breakpoint graph), one pair per amplicon. Needed for
  any question about structure: which segments, in what order, joined how.
- **Feature intervals as BED** — `*_classification_bed_files/`, one file per
  feature, the same intervals as `locations` in a form tools can read. Absent
  from some older archives.
- **Per-amplicon classification detail** —
  `*_amplicon_classification_profiles.tsv`, the scores and calls behind each
  feature's class, and `*_SV_summaries/`, the structural variants per amplicon.
- **Genome-wide copy number** — each sample's `*_CNV_CALLS.bed`, the calls
  used to find candidate regions, covering the whole genome rather than the
  amplified part. Present when the submitter included it, which is usual.
- **Figures** — AmpliconArchitect's amplicon plots as PNG and PDF.

The [AmpliconClassifier README](https://github.com/AmpliconSuite/AmpliconClassifier#3-outputs)
defines every file and column; read it before reading the archive. Find files
by name with a glob under `results/`: archives built by the current aggregator
put one gene list at `results/consolidated_classification/` and each sample's
cycles under `results/samples/<sample>/`, but projects submitted before the
layout was standardised keep them wherever the submitter's run wrote them,
sometimes one gene list per sample. Skip any match whose name starts with
`._` — a macOS resource fork some uploads carry, not data.

`/api/v1/projects/{id}/samples/` reports each sample's row of
AmpliconClassifier's results table under the same column names with spaces
replaced by underscores, so the results table itself never needs the download.

## Metadata is uneven, and this will change your answer

`classification`, `genes`, `oncogenes`, `locations`, `reference_build` and
the four feature numbers are computed by the analysis pipeline for every row
and are complete (the numbers are `null` where the pipeline recorded none,
which is almost always a row with nothing focal in it). Anything derived
from them is trustworthy.

`cancer_type`, `sample_type` and `tissue_of_origin` are **free text supplied by
whoever submitted each project**. They are not controlled vocabularies, they are
not unified across projects, and many rows have no value at all. Three
consequences, all of which produce confidently wrong answers if ignored:

1. **A filtered count is a floor, not an answer.** Compare `total_rows` in the
   facets response against the sum of the counts for the field you are
   filtering on. If they differ, the shortfall is rows carrying no value for
   that field, and no filter on it can ever reach them. Say so in your answer.
   Per project, `metadata_coverage` on the project object says the same thing
   directly: `"cancer_type": 0.0` means that project recorded no cancer type at
   all, so filtering it by cancer type returns an empty result that is not an
   answer.
2. **The same thing is spelled several ways.** Case is folded for you — `lung`
   and `Lung` are one entry with one count — but granularity is not: `Breast`
   and `Breast Adenocarcinoma` are separate values covering overlapping
   samples. Read the facet list and query the spellings you find there,
   repeating the parameter for each one you want.
3. **`NA` is a value, not a gap.** It means the submitter recorded "not
   applicable"; it is filterable and it is common.

Do not quote coverage figures from this file — there are none here on purpose,
because they change whenever a project's metadata is filled in. Compute them
from `/api/v1/features/facets/` at the time you answer.

## Calling conventions

- **No authentication is needed for public data.** A personal token additionally
  reaches private projects you are a member of, and raises rate limits. Send it
  as `Authorization: Token <token>`.
- **No browser User-Agent is needed.** Every `/api/v1/` path, plus `/robots.txt`
  and this file, answers any client. The HTML pages are a different matter and
  are not where you should be.
- **Gene lists are comma-delimited** (`gene_any=MYC,EGFR`). **Metadata values
  are not** — real ones contain commas, so repeat the parameter instead:
  `?cancer_type=Breast&cancer_type=Breast%20Adenocarcinoma`.
- **An unrecognised parameter is a `400`, not silence.** If you get one, the
  error names the parameters that do exist. Fix the name and retry rather than
  trusting a result you are not sure was filtered.
- **Errors are uniform**: `{"error": "...", "code": "..."}`. Branch on `code`,
  never on the prose in `error`.
- **Rate limits are per client.** On `429`, the response carries a `Retry-After`
  header and a `retry_after` field in seconds. Respect it — the limits exist
  because uncontrolled crawling has taken this site down before.
- **`403` means stop.** It is returned when you are authenticated but not a
  member of a private project; retrying with the same token will never succeed.
  `401` is the one that means "authenticate and try again".
- **`503` with code `index_unavailable` means retry shortly**, not "no results".
  The search index is rebuilding; an empty answer would be wrong.
- Read-only. The public API has no endpoint that changes anything.

## Please do not

- Crawl the project and sample **pages** to extract data that the API returns
  directly. See `/robots.txt`; the API is explicitly allowed there and bulk
  page-crawling is what the rate limits and load shedding exist to stop.
- Download archives in bulk. They are large, and `/features/` answers most
  questions.

## More

- [Documentation](https://docs.ampliconrepository.org/)
- [API reference](https://docs.ampliconrepository.org/en/latest/api/)
- [AmpliconSuite on GitHub](https://github.com/AmpliconSuite)
- [Contact](https://ampliconrepository.org/contact/)