Datasaur
Visit our websitePricingBlogPlaygroundAPI Docs
  • Welcome to Datasaur
    • Getting started with Datasaur
  • Data Studio Projects
    • Labeling Task Types
      • Span Based
        • OCR Labeling
        • Audio Project
      • Row Based
      • Document Based
      • Bounding Box
      • Conversational
      • Mixed Labeling
      • Project Templates
        • Test Project
    • Creating a Project
      • Data Formats
      • Data Samples
      • Split Files
      • Consensus
      • Dynamic Review Capabilities
    • Pre-Labeled Project
    • Let's Get Labeling!
      • Span Based
        • Span + Line Labeling
      • Row & Document Based
      • Bounding Box Labeling
      • Conversational Labeling
      • Label Sets / Question Sets
        • Dynamic Question Set
      • Multiple Label Sets
    • Reviewing Projects
      • Review Sampling
    • Adding Documents to an Ongoing Project
    • Export Project
  • LLM Projects
    • LLM Labs Introduction
    • Sandbox
      • Direct Access LLMs
      • File Attachment
      • Conversational Prompt
    • Deployment
      • Deployment API
    • Knowledge base
      • External Object Storage
      • File Properties
    • Models
      • Amazon SageMaker JumpStart
      • Amazon Bedrock
      • Open AI
      • Azure OpenAI
      • Vertex AI
      • Custom model
      • Fine-tuning
      • LLM Comparison Table
    • Evaluation
      • Automated Evaluation
        • Multi-application evaluation
        • Custom metrics
      • Ranking (RLHF)
      • Rating
      • Performance Monitoring
    • Dataset
    • Pricing Plan
  • Workspace Management
    • Workspace
    • Role & Permission
    • Analytics
      • Inter-Annotator Agreement (IAA)
        • Cohen's Kappa Calculation
        • Krippendorff's Alpha Calculation
      • Custom Report Builder
      • Project Report
      • Evaluation Metrics
    • Activity
    • File Transformer
      • Import Transformer
      • Export Transformer
      • Upload File Transformer
      • Running File Transformer
    • Label Management
      • Label Set Management
      • Question Set Management
    • Project Management
      • Self-Assignment
        • Self-Unassign
      • Transfer Assignment Ownership
      • Reset Labeling Work
      • Mark Document as Complete
      • Project Status Workflow
        • Read-only Mode
      • Comment Feature
      • Archive Project
    • Automation
      • Action: Create Projects
  • Assisted Labeling
    • ML Assisted Labeling
      • Amazon Comprehend
      • Amazon SageMaker
      • Azure ML
      • CoreNLP NER
      • CoreNLP POS
      • Custom API
      • FewNERD
      • Google Vertex AI
      • Hugging Face
      • LLM Assisted Labeling
        • Prompt Examples
        • Custom Provider
      • LLM Labs (beta)
      • NLTK
      • Sentiment Analysis
      • spaCy
      • SparkNLP NER
      • SparkNLP POS
    • Data Programming
      • Example of Labeling Functions
      • Labeling Function Analysis
      • Inter-Annotator Agreement for Data Programming
    • Predictive Labeling
  • Assisted Review
    • Label Error Detection
  • Building Your Own Model
    • Datasaur Dinamic
      • Datasaur Dinamic with Hugging Face
      • Datasaur Dinamic with Amazon SageMaker Autopilot
  • Advanced
    • Script-Generated Question
    • Shortcuts
    • Extensions
      • Labels
      • Review
      • Document and Row Labeling
      • Bounding Box Labels
      • List of Files
      • Comments
      • Analytics
      • Dictionary
      • Search
      • Labeling Guidelines
      • Metadata
      • Grammar Checker
      • ML Assisted Labeling
      • Data Programming
      • Datasaur Dinamic
      • Predictive Labeling
      • Label Error Detection
      • LLM Sandbox
    • Tokenizers
  • Integrations
    • External Object Storage
      • AWS S3
        • With IRSA
      • Google Cloud Storage
      • Azure Blob Storage
    • SAML
      • Okta
      • Microsoft Entra ID
    • SCIM
      • Okta
      • Microsoft Entra ID
    • Webhook Notifications
      • Webhook Signature
      • Events
      • Custom Headers
    • Robosaur
      • Commands
        • Create Projects
        • Apply Project Tags
        • Export Projects
        • Generate Time Per Task Report
        • Split Document
      • Storage Options
  • API
    • Datasaur APIs
    • Credentials
    • Create Project
      • New mutation (createProject)
      • Python Script Example
    • Adding Documents
    • Labeling
      • Create Label Set
      • Add Label Sets into Existing Project
      • Get List of Label Sets in a Project
      • Add Label Set Item into Project's Label Set
      • Programmatic API Labeling
      • Inserting Span and Arrow Label into Document
    • Export Project
      • Custom Webhook
    • Get Data
      • Get List of Projects
      • Get Document Information
      • Get List of Tags
      • Get Cabinet
      • Export Team Overview
      • Check Job
    • Custom OCR
      • Importable Format
    • Custom ASR
    • Run ML-Assisted Labeling
  • Security and Compliance
    • Security and Compliance
      • 2FA
  • Compatibility & Updates
    • Common Terminology
    • Recommended Machine Specifications
    • Supported Formats
    • Supported Languages
    • Release Notes
      • Version 6
        • 6.111.0
        • 6.110.0
        • 6.109.0
        • 6.108.0
        • 6.107.0
        • 6.106.0
        • 6.105.0
        • 6.104.0
        • 6.103.0
        • 6.102.0
        • 6.101.0
        • 6.100.0
        • 6.99.0
        • 6.98.0
        • 6.97.0
        • 6.96.0
        • 6.95.0
        • 6.94.0
        • 6.93.0
        • 6.92.0
        • 6.91.0
        • 6.90.0
        • 6.89.0
        • 6.88.0
        • 6.87.0
        • 6.86.0
        • 6.85.0
        • 6.84.0
        • 6.83.0
        • 6.82.0
        • 6.81.0
        • 6.80.0
        • 6.79.0
        • 6.78.0
        • 6.77.0
        • 6.76.0
        • 6.75.0
        • 6.74.0
        • 6.73.0
        • 6.72.0
        • 6.71.0
        • 6.70.0
        • 6.69.0
        • 6.68.0
        • 6.67.0
        • 6.66.0
        • 6.65.0
        • 6.64.0
        • 6.63.0
        • 6.62.0
        • 6.61.0
        • 6.60.0
        • 6.59.0
        • 6.58.0
        • 6.57.0
        • 6.56.0
        • 6.55.0
        • 6.54.0
        • 6.53.0
        • 6.52.0
        • 6.51.0
        • 6.50.0
        • 6.49.0
        • 6.48.0
        • 6.47.0
        • 6.46.0
        • 6.45.0
        • 6.44.0
        • 6.43.0
        • 6.42.0
        • 6.41.0
        • 6.40.0
        • 6.39.0
        • 6.38.0
        • 6.37.0
        • 6.36.0
        • 6.35.0
        • 6.34.0
        • 6.33.0
        • 6.32.0
        • 6.31.0
        • 6.30.0
        • 6.29.0
        • 6.28.0
        • 6.27.0
        • 6.26.0
        • 6.25.0
        • 6.24.0
        • 6.23.0
        • 6.22.0
        • 6.21.0
        • 6.20.0
        • 6.19.0
        • 6.18.0
        • 6.17.0
        • 6.16.0
        • 6.15.0
        • 6.14.0
        • 6.13.0
        • 6.12.0
        • 6.11.0
        • 6.10.0
        • 6.9.0
        • 6.8.0
        • 6.7.0
        • 6.6.0
        • 6.5.0
        • 6.4.0
        • 6.3.0
        • 6.2.0
        • 6.1.0
        • 6.0.0
      • Version 5
        • 5.63.0
        • 5.62.0
        • 5.61.0
        • 5.60.0
  • Deployment
    • Self-Hosted
      • AWS Marketplace
        • Data Studio
        • LLM Labs
Powered by GitBook
On this page
  • Overview
  • When to Use Script-Generated Questions
  • Configuration
  • Defining Question Identifiers
  • Dynamically Generating Questions
  • Limitation
  • Dropdown and Hierarchical Dropdown
  • Question Set Management
  • Question Logic
  1. Advanced

Script-Generated Question

Last updated 13 days ago

Overview

This feature only available in Row Labeling project and disabled by default. Please reach out to if your team needs this feature, and we'll assist you!

Script-Generated Questions is an advanced question type that allows dynamic question generation based on each row’s data. Using , users can configure logic that determines which questions should appear for a given row. This flexibility makes it ideal for cases where static question sets are insufficient.

When to Use Script-Generated Questions

  • You need dynamic questions that change based on row data.

  • You want to automate question creation instead of manually defining them.

  • You require custom logic to determine which questions appear.

  • You need dropdown options that are generated based on the data in each row.

Configuration

Defining Question Identifiers

An identifier is a unique representation of a question. It serves as a reference to the question type, label, and configuration, allowing the script to access and manipulate the question during execution. Before generating questions dynamically, all possible question identifiers must be defined during project creation.

This is configured in the identifiers part of the script.

Each identifier should have the following attributes:

  • id — A unique, zero-indexed integer, signifying the relative order of the questions.

  • label — A human-readable string that represents the question.

  • type — The question type (i.e. text, multiple choice, date).

  • config.multiple — Configuration that allows a question to have multiple answers.

Example Configuration

Here’s an example of an identifier for a text and a multiple-choice question.

identifiers: Record<string, QuestionIdentifier> = {
  ["Text Question"]: {
    id: 0, 
    label: "Text Question", 
    config: { multiple: false }, 
    type: QuestionType.TEXT
  },
  ["Multiple Choice Question"]: {
    id: 1, 
    label: "Multiple Choice Question", 
    config: { multiple: true }, 
    type: QuestionType.MULTIPLE_CHOICE
  },
};

Dynamically Generating Questions

The core function that determines which questions appear is getQuestionsForRow. This function runs for each row in the dataset, evaluating its data and returning the appropriate questions.

Tip: You can view available row and columns attributes in the editor by pressing Ctrl / Cmd + Space.

Example Usage

In the following example:

  • A text question appears if the column DATA contains the word "text".

  • Otherwise, a multiple-choice question is shown, with options generated from the values in DATA.

getQuestionsForRow({ row, columns }: RowContext): Question[] {
  if (getCellValueByColumnLabel("DATA").includes("text")) {
    return [{ ...this.identifiers["Text Question"], required: false }];
  }

  const options = getCellValueByColumnLabel("DATA").split(",");
  return [
    {
      ...this.identifiers["Multiple Choice Question"],
      required: false,
      config: {
        ...this.identifiers["Multiple Choice Question"],
        options: options.map((o) => ({ id: o, label: o })),
      },
    },
  ];

Here’s how it will look once the project is created.

You can find a sample CSV, as well as the full script to recreate the project below.

Limitation

Dropdown and Hierarchical Dropdown

We currently don’t support configuring Dropdown or Hierarchical dropdown question types in Script-Generated Questions.

We recommend using Multiple Choice or Single Choice question types, as these also allow users to select from a set of options.

Question Set Management

  1. Any Script-Generated Questions will not be saved to the library when the question set is saved.

  2. When configuring the question set in Label Set Management, these questions will not appear in the question type dropdown.

Question Logic

Since question logic relies on a predefined order, it’s not currently possible to configure logic for questions that reference a script-generated one. This is because Script-Generated Questions are created dynamically during the project, which makes it difficult to set conditions for subsequent questions based on them.

What this means is that questions following a script-generated one can still have logic, but they can only reference other question types, not the script-generated question itself.

Currently, all question types are supported except Grouped Attributes and Script-Generated Questions, with .

See for more details on each question type.

some caveat
support@datasaur.ai
TypeScript
this
651B
Datasaur sample - Script generated question.csv
3KB
Datasaur sample - Script for row question.ts
Identifiers