Skip to content

Commit

Permalink
docs(faq): reorganise by category (#2649)
Browse files Browse the repository at this point in the history
Break the FAQ into 4 broad categories:

- background
- usage
- data
- infrastructure

and place the existing entries under these.

Address various markdownlint complaints and a config file to disable
- line length
- embedded HTML
  • Loading branch information
andrewpollock authored Sep 25, 2024
1 parent 576b1b3 commit 89d880a
Show file tree
Hide file tree
Showing 2 changed files with 84 additions and 65 deletions.
5 changes: 5 additions & 0 deletions .markdownlint.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
{
"MD013": false,
"MD025": false,
"MD033": false
}
144 changes: 79 additions & 65 deletions docs/faq.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,8 @@ title: FAQ
permalink: /faq/
nav_order: 4
---
# FAQ

# Frequently Asked Questions

{: .no_toc }

Expand All @@ -17,7 +18,9 @@ nav_order: 4
{:toc}
</details>

## What is OSV?
## Background

### What is OSV?

OSV consists of:

Expand All @@ -38,14 +41,15 @@ See our blog posts for more details:
2. [Announcing a unified vulnerability schema for open source](https://security.googleblog.com/2021/06/announcing-unified-vulnerability-schema.html)
3. [OSV and the Vulnerability Life Cycle](https://security.googleblog.com/2023/03/osv-and-vulnerability-life-cycle.html)

## Who is OSV for?
### Who is OSV for?

The OSV schema and OSV.dev can be used by both:
The OSV schema and OSV.dev can be used by:

1. Open source consumers: By querying [OSV.dev's API](../api/) and using our tooling to find known vulnerabilities in their dependencies.
2. Vulnerability database producers: By making the database available in the OSV format.
2. Open source projects: By publishing vulnerabilities in the OSV format and having them imported by OSV.dev.
3. Vulnerability database producers: By making the database available in the OSV format.

## Why a new format to describe vulnerability?
### Why a new format to describe vulnerability?

We found that there was no existing standard format which:

Expand All @@ -55,105 +59,115 @@ We found that there was no existing standard format which:

A unified format means that vulnerability databases, open source users, and security researchers can easily share tooling and consume vulnerabilities across all of open source. This means a more complete view of vulnerabilities in open source for everyone, as well as faster detection and remediation times resulting from easier automation.

## Who is using the OSV schema?
### Who is using the OSV schema?

The benefits of the OSV schema have led to adoption by several vulnerability databases, including GitHub Security Advisories, PyPA, RustSec, and many more. The full list of databases can be found [here](https://github.com/ossf/osv-schema#open-source-vulnerability-schema).

## What does OSV.dev do to the records it imports?
## Usage

1. Version enumeration (for non-SemVer ecosystems where [supporting version enumeration code](https://github.com/google/osv.dev/tree/master/osv/ecosystems) exists)
2. [Package URL](https://github.com/package-url/purl-spec) [computation](https://github.com/google/osv.dev/blob/a751ceb26522f093edf26c0ad167cfd0967716d9/osv/models.py#L361-L365) (if necessary)
3. [Git affected commit enumeration and commit to tag mapping](https://github.com/google/osv.dev/blob/a751ceb26522f093edf26c0ad167cfd0967716d9/osv/impact.py#L422)
4. [Batch](https://github.com/google/osv.dev/blob/master/deployment/clouddeploy/gke-workers/base/alias-computation.yaml) [computation](https://github.com/google/osv.dev/tree/master/docker/alias) of [aliases](https://ossf.github.io/osv-schema/#aliases-field)
### How do I use OSV as an open source user?

Both version and commit enumeration populate the [`affected.versions[]`](https://ossf.github.io/osv-schema/#affectedversions-field) field, which assists with precise version matching.
OSV.dev provides an [easy-to-use API](../api/) for querying against the aggregated database of vulnerabilities.

In some cases, there may be no applicable versions, so the `affected.versions[]`
array is empty. This field, when empty, is omitted in the API output, and
present (but empty) in the [data exports](#is-the-database-available-to-download).
[Command line tooling](https://github.com/google/osv-scanner) is also available for vulnerability scanning of SBOMs, language manifests, and container images.

## How does OSV.dev handle withdrawn records?
There are numerous third-party integrations with the API, and we have stopped maintaining an exhaustive list. You can try [this GitHub search](https://github.com/search?q=%22api.osv.dev%22&type=code) as a starting point.

Records that have the [`withdrawn`](https://ossf.github.io/osv-schema/#withdrawn-field) field set will be excluded from:
* the responses to POST API queries
* the main [list page](https://osv.dev/list) and related search results
### How do I use OSV as a vulnerability database maintainer?

The entry remains in the database, and is:
* returned by the `/vulns/<ID>` GET API
* visible at `https://osv.dev/vulnerability/<ID>` page (and clearly visibly marked as "withdrawn")
* still exported in the [GCS exports](#is-the-database-available-to-download) (including the `withdrawn` field)
By [making your vulnerability database available in the OSV format](https://github.com/google/osv.dev/blob/master/CONTRIBUTING.md#contributing-data), open source users will have a consistent way to consume vulnerabilities across all open source ecosystems.

Vulnerability databases can also benefit from easier interchange and vulnerability sharing from other databases that use the OSV format.

## How does OSV.dev handle deleted records?
### How do I contribute to OSV, or ask a question?

When a record is deleted from an upstream source, OSV.dev currently handles them differently, depending on where they're imported from:
OSV is completely open source!

- GCS sources: OSV.dev marks deleted records as [`withdrawn`](https://ossf.github.io/osv-schema/#withdrawn-field). There is additionally a safety threshold in the case of feed availability issues: if more than 10% of records are about to be marked as `withdrawn`, OSV.dev aborts and does not proceed.
- REST and Git sources: OSV.dev leaves the existing records valid but orphaned. This behaviour will be changed to match the GCS source. See <https://github.com/google/osv.dev/issues/2101> and <https://github.com/google/osv.dev/issues/2294>.
1. The infrastructure code is available [here](https://github.com/google/osv.dev)
2. OSV-Scanner code is available [here](https://github.com/google/osv-scanner)
3. The OSV schema spec is available [here](https://github.com/ossf/osv-schema)

## How do I use OSV as an open source user?
If you have any questions, please feel free to create an issue!

OSV.dev provides an [easy-to-use API](../api/) for querying against the aggregated database of vulnerabilities.
## Data

[Command line tooling](https://github.com/google/osv-scanner) is also available for vulnerability scanning of SBOMs, language manifests, and container images.
### I've found something wrong with the data

## How do I use OSV as a vulnerability database maintainer?
Data quality is very important to us. Please remember that OSV.dev is an
aggregator of OSV records from a [variety of
sources](https://github.com/google/osv.dev/blob/master/source.yaml) and the most
appropriate place to correct the data is at the source.

By [making your vulnerability database available in the OSV format](https://github.com/google/osv.dev/blob/master/CONTRIBUTING.md#contributing-data), open source users will have a consistent way to consume vulnerabilities across all open source ecosystems.
We prefer to avoid needing to act as a broker between downstream consumers of
the data and upstream sources, as this adds limited value, and only adds delays.

Vulnerability databases can also benefit from easier interchange and vulnerability sharing from other databases that use the OSV format.
Where available, a human-friendly link to the authoritative record source is
available as the `Source` field on the individual vulnerability page. You should
follow the source-specific process for updating the data.

For sources that are a Git repository, the `Import Source` field points to the
authoritative source of the data, and you may be able to create a pull/merge
request or file an issue against the repository.

## Is the database available to download?
If you are not able to get satisfaction after dealing directly with the source of the data, please [file an issue](https://github.com/google/osv.dev/issues?q=is%3Aissue+is%3Aopen+label%3A%22data+quality%22) tagged with `data quality`.

### Is the database available to download?

Yes!

The database in available in a GCS bucket maintained by OSV: [gs://osv-vulnerabilities](https://storage.googleapis.com/osv-vulnerabilities/index.html) (also [publicly browseable via the Google Cloud Console](https://console.cloud.google.com/storage/browser/osv-vulnerabilities) with a login)

More information about how to download the database is available [here](data.md#data-dumps).

## What are OSV's service level objectives (SLOs)?
### Can I contribute data?

Yes!

OSV strives to provide reliable vulnerability information to our users. To support that goal, target the following service level objectives:
If you work on a project (like a Linux distribution) and would like to contribute security advisories, please see our data contribution [guide](https://github.com/google/osv.dev/blob/master/CONTRIBUTING.md#contributing-data) on GitHub.

1. Availability, website and API: 99.9% measured on a 7 day rolling window.
2. Latency, website and API: P50 ≤ 300ms, P90 ≤ 500ms, P95 ≤ 1s, that is 50% of requests will be faster than 300ms, 90% of requests will be faster than 500ms, and 95% of requests will be faster than 1s.
3. Data Freshness: Data sources no more than 15 minutes stale, 99.5% of the time.
### What does OSV.dev do to the records it imports?

## How do I contribute to OSV, or ask a question?
1. Version enumeration (for non-SemVer ecosystems where [supporting version enumeration code](https://github.com/google/osv.dev/tree/master/osv/ecosystems) exists)
2. [Package URL](https://github.com/package-url/purl-spec) [computation](https://github.com/google/osv.dev/blob/a751ceb26522f093edf26c0ad167cfd0967716d9/osv/models.py#L361-L365) (if necessary)
3. [Git affected commit enumeration and commit to tag mapping](https://github.com/google/osv.dev/blob/a751ceb26522f093edf26c0ad167cfd0967716d9/osv/impact.py#L422)
4. [Batch](https://github.com/google/osv.dev/blob/master/deployment/clouddeploy/gke-workers/base/alias-computation.yaml) [computation](https://github.com/google/osv.dev/tree/master/docker/alias) of [aliases](https://ossf.github.io/osv-schema/#aliases-field)

OSV is completely open source!
Both version and commit enumeration populate the [`affected.versions[]`](https://ossf.github.io/osv-schema/#affectedversions-field) field, which assists with precise version matching.

1. The infrastructure code is available [here](https://github.com/google/osv.dev)
2. OSV-Scanner code is available [here](https://github.com/google/osv-scanner)
3. The OSV schema spec is available [here](https://github.com/ossf/osv-schema)
In some cases, there may be no applicable versions, so the `affected.versions[]`
array is empty. This field, when empty, is omitted in the API output, and
present (but empty) in the [data exports](#is-the-database-available-to-download).

If you have any questions, please feel free to create an issue!
### How does OSV.dev handle withdrawn records?

## Can I contribute data?
Records that have the [`withdrawn`](https://ossf.github.io/osv-schema/#withdrawn-field) field set will be excluded from:

Yes!
* the responses to POST API queries
* the main [list page](https://osv.dev/list) and related search results

If you work on a project (like a Linux distribution) and would like to contribute security advisories, please see our data contribution [guide](https://github.com/google/osv.dev/blob/master/CONTRIBUTING.md#contributing-data) on GitHub.
The entry remains in the database, and is:

## Is the API rate limited?
* returned by the `/vulns/<ID>` GET API
* visible at `https://osv.dev/vulnerability/<ID>` page (and clearly visibly marked as "withdrawn")
* still exported in the [GCS exports](#is-the-database-available-to-download) (including the `withdrawn` field)

No. Currently there is not a limit on the API.
### How does OSV.dev handle deleted records?

## I've found something wrong with the data
When a record is deleted from an upstream source, OSV.dev currently handles them differently, depending on where they're imported from:

Data quality is very important to us. Please remember that OSV.dev is an
aggregator of OSV records from a [variety of
sources](https://github.com/google/osv.dev/blob/master/source.yaml) and the most
appropriate place to correct the data is at the source.
* GCS sources: OSV.dev marks deleted records as [`withdrawn`](https://ossf.github.io/osv-schema/#withdrawn-field). There is additionally a safety threshold in the case of feed availability issues: if more than 10% of records are about to be marked as `withdrawn`, OSV.dev aborts and does not proceed.
* REST and Git sources: OSV.dev leaves the existing records valid but orphaned. This behaviour will be changed to match the GCS source. See <https://github.com/google/osv.dev/issues/2101> and <https://github.com/google/osv.dev/issues/2294>.

We prefer to avoid needing to act as a broker between downstream consumers of
the data and upstream sources, as this adds limited value, and only adds delays.
## Infrastructure

Where available, a human-friendly link to the authoritative record source is
available as the `Source` field on the individual vulnerability page. You should
follow the source-specific process for updating the data.
### Is the API rate limited?

For sources that are a Git repository, the `Import Source` field points to the
authoritative source of the data, and you may be able to create a pull/merge
request or file an issue against the repository.
No. Currently there is not a limit on the API.

If you are not able to get satisfaction after dealing directly with the source of the data, please [file an issue](https://github.com/google/osv.dev/issues?q=is%3Aissue+is%3Aopen+label%3A%22data+quality%22) tagged with `data quality`.
### What are OSV.dev's service level objectives (SLOs)?

OSV.dev strives to provide reliable vulnerability information to our users. To support that goal, the following service level objectives are targeted:

1. Availability, website and API: 99.9% measured on a 7 day rolling window.
2. Latency, website and API: P50 ≤ 300ms, P90 ≤ 500ms, P95 ≤ 1s, that is 50% of requests will be faster than 300ms, 90% of requests will be faster than 500ms, and 95% of requests will be faster than 1s.
3. Data Freshness: Data sources no more than 15 minutes stale, 99.5% of the time.

0 comments on commit 89d880a

Please sign in to comment.