Datasaur
Visit our websitePricingBlogPlaygroundAPI Docs
  • Welcome to Datasaur
    • Getting started with Datasaur
  • Data Studio Projects
    • Labeling Task Types
      • Span Based
        • OCR Labeling
        • Audio Project
      • Row Based
      • Document Based
      • Bounding Box
      • Conversational
      • Mixed Labeling
      • Project Templates
        • Test Project
    • Creating a Project
      • Data Formats
      • Data Samples
      • Split Files
      • Consensus
      • Dynamic Review Capabilities
    • Pre-Labeled Project
    • Let's Get Labeling!
      • Span Based
        • Span + Line Labeling
      • Row & Document Based
      • Bounding Box Labeling
      • Conversational Labeling
      • Label Sets / Question Sets
        • Dynamic Question Set
      • Multiple Label Sets
    • Reviewing Projects
      • Review Sampling
    • Adding Documents to an Ongoing Project
    • Export Project
  • LLM Projects
    • LLM Labs Introduction
    • Sandbox
      • Direct Access LLMs
      • File Attachment
      • Conversational Prompt
    • Deployment
      • Deployment API
    • Knowledge base
      • External Object Storage
      • File Properties
      • Chunk Editor
    • Models
      • Amazon SageMaker JumpStart
      • Amazon Bedrock
      • Open AI
      • Azure OpenAI
      • Vertex AI
      • Custom model
      • Fine-tuning
      • LLM Comparison Table
    • Evaluation
      • Automated Evaluation
        • Multi-application evaluation
        • Custom metrics
      • Ranking (RLHF)
      • Rating
      • Performance Monitoring
    • Dataset
    • Pricing Plan
  • Workspace Management
    • Workspace
    • Role & Permission
    • Analytics
      • Inter-Annotator Agreement (IAA)
        • Cohen's Kappa Calculation
        • Krippendorff's Alpha Calculation
      • Custom Report Builder
      • Project Report
      • Evaluation Metrics
    • Activity
    • File Transformer
      • Import Transformer
      • Export Transformer
      • Upload File Transformer
      • Running File Transformer
    • Label Management
      • Label Set Management
      • Question Set Management
    • Project Management
      • Self-Assignment
        • Self-Unassign
      • Transfer Assignment Ownership
      • Reset Labeling Work
      • Mark Document as Complete
      • Project Status Workflow
        • Read-only Mode
      • Comment Feature
      • Archive Project
    • Automation
      • Action: Create Projects
  • Assisted Labeling
    • ML Assisted Labeling
      • Amazon Comprehend
      • Amazon SageMaker
      • Azure ML
      • CoreNLP NER
      • CoreNLP POS
      • Custom API
      • FewNERD
      • Google Vertex AI
      • Hugging Face
      • LLM Assisted Labeling
        • Prompt Examples
        • Custom Provider
      • LLM Labs (beta)
      • NLTK
      • Sentiment Analysis
      • spaCy
      • SparkNLP NER
      • SparkNLP POS
    • Data Programming
      • Example of Labeling Functions
      • Labeling Function Analysis
      • Inter-Annotator Agreement for Data Programming
    • Predictive Labeling
  • Assisted Review
    • Label Error Detection
  • Building Your Own Model
    • Datasaur Dinamic
      • Datasaur Dinamic with Hugging Face
      • Datasaur Dinamic with Amazon SageMaker Autopilot
  • Advanced
    • Script-Generated Question
    • Shortcuts
    • Extensions
      • Labels
      • Review
      • Document and Row Labeling
      • Bounding Box Labels
      • List of Files
      • Comments
      • Analytics
      • Dictionary
      • Search
      • Labeling Guidelines
      • Metadata
      • Grammar Checker
      • ML Assisted Labeling
      • Data Programming
      • Datasaur Dinamic
      • Predictive Labeling
      • Label Error Detection
      • LLM Sandbox
    • Tokenizers
  • Integrations
    • External Object Storage
      • AWS S3
        • With IRSA
      • Google Cloud Storage
      • Azure Blob Storage
      • Dropbox
    • SAML
      • Okta
      • Microsoft Entra ID
    • SCIM
      • Okta
      • Microsoft Entra ID
    • Webhook Notifications
      • Webhook Signature
      • Events
      • Custom Headers
    • Robosaur
      • Commands
        • Create Projects
        • Apply Project Tags
        • Export Projects
        • Generate Time Per Task Report
        • Split Document
      • Storage Options
  • API
    • Datasaur APIs
    • Credentials
    • Create Project
      • New mutation (createProject)
      • Python Script Example
    • Adding Documents
    • Labeling
      • Create Label Set
      • Add Label Sets into Existing Project
      • Get List of Label Sets in a Project
      • Add Label Set Item into Project's Label Set
      • Programmatic API Labeling
      • Inserting Span and Arrow Label into Document
    • Export Project
      • Custom Webhook
    • Get Data
      • Get List of Projects
      • Get Document Information
      • Get List of Tags
      • Get Cabinet
      • Export Team Overview
      • Check Job
    • Custom OCR
      • Importable Format
    • Custom ASR
    • Run ML-Assisted Labeling
  • Security and Compliance
    • Security and Compliance
      • 2FA
  • Compatibility & Updates
    • Common Terminology
    • Recommended Machine Specifications
    • Supported Formats
    • Supported Languages
    • Release Notes
      • Version 6
        • 6.113.0
        • 6.112.0
        • 6.111.0
        • 6.110.0
        • 6.109.0
        • 6.108.0
        • 6.107.0
        • 6.106.0
        • 6.105.0
        • 6.104.0
        • 6.103.0
        • 6.102.0
        • 6.101.0
        • 6.100.0
        • 6.99.0
        • 6.98.0
        • 6.97.0
        • 6.96.0
        • 6.95.0
        • 6.94.0
        • 6.93.0
        • 6.92.0
        • 6.91.0
        • 6.90.0
        • 6.89.0
        • 6.88.0
        • 6.87.0
        • 6.86.0
        • 6.85.0
        • 6.84.0
        • 6.83.0
        • 6.82.0
        • 6.81.0
        • 6.80.0
        • 6.79.0
        • 6.78.0
        • 6.77.0
        • 6.76.0
        • 6.75.0
        • 6.74.0
        • 6.73.0
        • 6.72.0
        • 6.71.0
        • 6.70.0
        • 6.69.0
        • 6.68.0
        • 6.67.0
        • 6.66.0
        • 6.65.0
        • 6.64.0
        • 6.63.0
        • 6.62.0
        • 6.61.0
        • 6.60.0
        • 6.59.0
        • 6.58.0
        • 6.57.0
        • 6.56.0
        • 6.55.0
        • 6.54.0
        • 6.53.0
        • 6.52.0
        • 6.51.0
        • 6.50.0
        • 6.49.0
        • 6.48.0
        • 6.47.0
        • 6.46.0
        • 6.45.0
        • 6.44.0
        • 6.43.0
        • 6.42.0
        • 6.41.0
        • 6.40.0
        • 6.39.0
        • 6.38.0
        • 6.37.0
        • 6.36.0
        • 6.35.0
        • 6.34.0
        • 6.33.0
        • 6.32.0
        • 6.31.0
        • 6.30.0
        • 6.29.0
        • 6.28.0
        • 6.27.0
        • 6.26.0
        • 6.25.0
        • 6.24.0
        • 6.23.0
        • 6.22.0
        • 6.21.0
        • 6.20.0
        • 6.19.0
        • 6.18.0
        • 6.17.0
        • 6.16.0
        • 6.15.0
        • 6.14.0
        • 6.13.0
        • 6.12.0
        • 6.11.0
        • 6.10.0
        • 6.9.0
        • 6.8.0
        • 6.7.0
        • 6.6.0
        • 6.5.0
        • 6.4.0
        • 6.3.0
        • 6.2.0
        • 6.1.0
        • 6.0.0
      • Version 5
        • 5.63.0
        • 5.62.0
        • 5.61.0
        • 5.60.0
  • Deployment
    • Self-Hosted
      • AWS Marketplace
        • Data Studio
        • LLM Labs
Powered by GitBook
On this page
  • Overview
  • Get started
  • Understanding roles
  • Write conversation manually
  • Creating the JSON file format
  • Key Differences: Conversational vs Basic Prompt
  1. LLM Projects
  2. Sandbox

Conversational Prompt

Last updated 1 month ago

Overview

Conversational Prompting brings the power of multi-turn conversations testing to LLM Labs. This feature allows users to simulate real-world conversational scenarios, ensuring that language models maintain context and coherence across multiple interactions. Designed to enhance the experimentation process, Conversational Prompting provides a seamless and intuitive way to interact with and assess the performance of language models in a conversational context.

Get started

To begin using the Conversational prompt feature:

  1. Navigate to your .

  2. Click the three-dots button located in a prompt of the Sandbox interface.

  3. Choose your method:

    • Add conversation: Manually input a new conversation directly into the Sandbox.

    • Upload conversation file (.json): Upload a pre-defined conversation file in JSON format for testing.

  4. Execute the conversation and observe how the language model performs, maintaining context and coherence across turns.

Use the role dropdown in the prompt section to change the role of the assistant or user as needed for your testing scenario.

Understanding roles

In general, roles are used to represent the conversational input from the user and the assistant.

  • User: Represents input from the user, such as a request or question sent to the application.

  • Assistant: Represents the application’s response, either as a reply or part of the conversation history in a prompt.

You can manually change the roles via the dropdown when writing the conversational prompt to define the prompt input. This flexibility allows you to simulate various conversational dynamics and test how the model responds to different types of interactions.

Write conversation manually

  1. In the Add conversation dialog:

    1. Write the prompt to represent the input based on your needs, whether it is for the assistant or user roles.

    2. Use the role dropdown to specify the role for each message (e.g., user or assistant).

    3. Enter the content of the message in the provided text box.

  2. Adding more conversations:

    • You can also add more conversation in this dialog. To add the conversation, simply hover to the text input, and the Add above and Add below buttons will appear. The differences between both buttons are:

      1. When you click the Add above button, a new text input will appear above the current text input that you are hovering.

      2. When you click the Add below button, a new text input will appear under the current text input that you are hovering.

      3. You can also click the Delete button to remove the text input.

  3. Reorder the conversation: You can also drag and drop the conversation input to reorder it based on your needs. This feature allows you to easily rearrange the sequence of messages to better simulate different conversational flows.

  4. Attach files and URLs:

    • Please note that this capability is only available for the User role; the Assistant role cannot attach any files or URLs.

    • Each message in a conversation can have up to 5 attachments. For example, with 3 messages in a prompt, you can attach up to 15 files.

Creating the JSON file format

Structure:

The core structure is an array of objects:

[
  {
    "content": "...",
    "role": "..."
  },
  {
    "content": "...",
    "role": "..."
  },
  // ... more turns
]

Fields:

  • content:

    • This field holds the actual message or interaction data.

    • It can be a simple string for plain text messages.

    • Alternatively, it can be an array of objects to include attachments like URLs or images.

    • Important: This field cannot be empty. Every conversation turn must contain content.

  • role:

    • This field indicates the speaker's role in the conversation.

    • It can be either "user" or "assistant".

Attachments (URLs and files):

To include attachments, the content field should be an array of objects. Each object within this array can represent text, a file URL, or a website URL.

  • Text:

    • Use {"type": "text", "text": "Your text here"}.

  • File URL:

    • Use {"type": "image_url", "image_url": {"url": "<https://example.com/image.jpg>"}}.

  • URL (Website Link):

    • Use {"type": "image_url", "url": "<https://example.com>"}.

Sample JSON file:

Key Differences: Conversational vs Basic Prompt

Conversational Prompt

  • Maintains and understands the full history of the conversation.

  • Handles back-and-forth dialogue naturally.

  • Generates replies based on accumulated context.

Basic Prompt

  • Treats each input as an independent request.

  • Has no memory of previous interactions.

  • Generates replies based solely on the immediate input.

You can attach a file or URLs in the conversation input, but only for the User role. This capability is useful for simulating real-world scenarios where users might share documents or links. .

Learn more about file attachment
Sandbox
Learn more on how to write the conversation.
Learn more about the JSON file format.
1KB
Conversational Prompt Sample.json