Skip to main content

Scoring profiles: reproducible decisions for every job

· 3 min read
DeepFAIC Team
Engineering & Product

Make scoring changes deliberate with signed bundles, explicit profile selection, and versions that stay attached to each Edge job.

When a team's review policy changes, yesterday's analysis should remain understandable. Reproducibility means being able to say which configuration produced a decision, even after the deployment has moved on.

DeepFAIC Edge scoring profiles package a tuned scoring configuration into a signed bundle. The configuration can include model weights, decision thresholds, and custom scoring logic authored and tested by the DeepFAIC team for a use case.

Importing is separate from selecting​

Importing a profile does not change the scoring of every job. Profiles are opt-in per request. Jobs without a profile_name continue to use the deployment's CONFIG.yaml defaults.

That separation is useful during rollout. Your team can install a bundle, confirm it appears in the profile list, and evaluate it through explicitly selected jobs before changing application behavior.

The runtime verifies the bundle's signature during import. A modified file or a bundle that was not produced by DeepFAIC is rejected. This is especially useful for deployments that receive updates through an offline file transfer process.

Pin a version when consistency matters​

There are two ways to select a profile:

SelectorBehavior
clip-jurySelect the latest installed version of the named profile
clip-jury@3Select that specific version

These are example names from the documentation, not profiles included automatically with every deployment.

For a controlled evaluation or a policy-sensitive workflow, pinning a version makes the application's intent explicit. Selecting the latest version is useful when your team deliberately wants new jobs to follow the newest installed configuration.

The job keeps its scoring identity​

The selected profile is frozen onto the job at creation. Installing a newer profile version does not change an existing job's results. The response reports prediction_source, such as clip-jury@3, so downstream systems can record what produced the prediction.

Keep this identifier alongside your application submission ID and review outcome. If a reviewer asks why two submissions were treated differently, you can check whether their inputs, analysis status, and scoring configurations were comparable.

Roll out with a comparison set​

Before directing a workflow to a new profile, choose a set of representative media and a clear evaluation policy. Run comparable jobs using the current and candidate configurations, then inspect the disagreements.

Avoid judging a candidate only by how many items it flags. Review false positives, missed cases, failed analysis, and the resulting review workload. The right configuration depends on the action your product takes after analysis.

Retire without rewriting history​

Profile removal is soft. New jobs can no longer select the removed version, while jobs that already used it keep their results. Re-importing the same bundle can reactivate it.

Signed delivery, explicit selection, and a recorded version make configuration changes reviewable. They let a deployment evolve while preserving a clear explanation of past decisions.

Read the scoring profile reference for import, list, selection, and removal details.