Spyglass

Spyglass

Spyglass is a pluggable artifact viewer framework for Prow. It collects artifacts (usually files in a storage bucket) from various sources and distributes them to registered viewers, which are responsible for consuming them and rendering a view.

A typical Spyglass page might look something like this:

If you want to know how to write a Spyglass lens, check the lens-writing guide. If you’re interested in how Spyglass works, check the architecture summary.

Configuration

Using Spyglass on your Prow instance requires you to first enable Spyglass in deck, and then configure Spyglass to actually do something.

Enabling Spyglass

To enable spyglass, just pass the --spyglass flag to your deck instance. Once spyglass is enabled, it will expose itself under /view/ on your deck instance.

In order to make Spyglass useful, you may want to set your job URLs to point at it. You can do so by setting plank.job_url_prefix_config['*'] to https://your.deck/view/, and possibly plank.job_url_template to reference something similar depending on your setup.

If you are not using the images we provide, you may also need to provide --spyglass-files-location, pointing at the on-disk location of the lenses folder in this directory.

Configuring Spyglass

Spyglass configuration is contained in the spyglass subsection of the deck section of Prow’s primary configuration.

The spyglass block has the following properties:

Name Required Example Description
size_limit Yes 100000000 The maximum size of an artifact to download, in bytes. Larger values will be omitted or truncated.
gcs_browser_prefix No https://gcsweb.k8s.io/gcs/
https://s3.console.aws.amazon.com/s3/buckets/
If you have a GCS browser available, the bucket and path to the artifact directory will be appended to gcs_browser_prefix and linked from Spyglass pages. If left unset, no artifacts link will be visible. The provided URL should have a trailing slash
testgrid_config No gs://k8s-testgrid/config If you have a TestGrid instance available, testgrid_config should point to the TestGrid config proto on GCS. If omitted, no TestGrid link will be visible.
testgrid_root No https://testgrid.k8s.io/ If you have a TestGrid instance available, testgrid_root should point to the root of the TestGrid web interface. If omitted, no TestGrid link will be visible.
announcement No "Remember: friendship is magic!" If announcement is set, the string will appear at the top of the page. announcement is parsed as a Go template. The only value provided is .ArtifactPath, which is of the form gcs-bucket/path/to/job/root/.
lenses Yes (see below) lenses configures the lenses you want, when they should be visible, what artifacts they should receive, and any lens specific configuration

Configuring Lenses

Lenses are the Spyglass components that actually display information. The lenses block under the spyglass block is a list of configuration for each lens. Each lens entry has the following properties:

Name Required Example Description
required_files Yes - build-log\.txt A list of regexes matching artifact names that must be present for a lens to appear. The list entries are ANDed together - that is, something much match every entry. OR can be simulated by using a pipe in a single regex entry.
optional_files No - something\.txt A list of regexes matching artifact names that will be provided to a lens if present, but are not necessary for it to appear (for that, use required_files). Since each entry in the list is optional, these are effectively ORed together.
lens.name Yes buildlog The name of the lens you want to render these files. Must be a known lens name.
lens.config No Lens-specific configuration. What can be included here, if anything, depends on the lens in question.

The following lenses are available:

  • metadata: parses the metadata files generated by podutils and displays their content. It has no configuration.
  • junit: parses junit files and displays their content. It has no configuration
  • buildlog: displays the build log (or any other log file), highlighting interesting parts and hiding the rest behind expandable folders. You can configure what it considers “interesting” by providing highlight_regexes, a list of regexes to highlight. If not specified, it uses defaults optimised for highlighting Kubernetes test results. The optional hide_raw_log boolean field can be used to omit the link to the raw build-log.txt source.
  • podinfo: displays info about ProwJob pods including the events and details about containers and volumes. The gcsk8sreporter Crier reporter must be enabled to upload the required podinfo.json file.
  • coverage: displays go coverage content
  • restcoverage: displays REST API statistics

Example Configuration

deck:
  spyglass:
    size_limit: 100000000  # 100 MB
    gcs_browser_prefix: https://gcsweb.k8s.io/gcs/
    testgrid_config: gs://k8s-testgrid/config
    testgrid_root: https://testgrid.k8s.io/
    announcement: "The old job viewer has been deprecated."
    lenses:
    - lens:
        name: metadata
      required_files:
      - ^(?:started|finished)\.json$
      optional_files:
      - ^(?:podinfo|prowjob)\.json$
    - lens:
        name: buildlog
        config:
          highlight_regexes:
          - timed out
          - 'ERROR:'
          - (FAIL|Failure \[)\b
          - panic\b
          - ^E\d{4} \d\d:\d\d:\d\d\.\d\d\d]
      required_files:
      - ^build-log\.txt$
    - lens:
        name: junit
      required_files:
      - ^artifacts/junit.*\.xml$
    - lens:
        name: podinfo
        config:
          runner_configs: # Would only work if `prowjob.json` is configured below
            "<BUILD_CLUSTER_ALIAS>":
              pod_link_template: "https://<YOUR_CLOUD_PROVIDER_URL>/{{ .Name }}" # Name is directly from the Pod struct.
            # Example:
            # "default":
            #    pod_link_template: "https://console.cloud.google.com/kubernetes/pod/us-central1-f/prow/test-pods/{{ .Name }}/details?project=k8s-prow-builds"
      required_files:
        - ^podinfo\.json$
      optional_files:
        - ^prowjob\.json$ # Only if runner_configs is configured.

Accessing custom storage buckets

By default, spyglass has access to all storage buckets defined globally (plank.default_decoration_config_entries[...].gcs_configuration) or on individual jobs (<path-to-job>.gcs_configuration.bucket). In order to access additional/custom storage buckets, those buckets must be listed in deck.additional_storage_buckets.