Version 6

There are breaking changes related to Row Labeling and Document Labeling since we improve the performance by updating the database schema, which significantly make the Row and Document Labeling faster, especially when answering a question.

For any self-hosted users, please don't hesitate to contact us at [e-mail]. We will help you upgrade to this major version.

6.88.0

November 14, 2024

What's new 💡

NLP

• Label set dialog in project — Enhanced the appearance of the label set dialog for configuring new or existing label sets within a project.

• ML Assisted Labeling extension — Enabled by default for Span and Row Labeling projects in both Labeler and Reviewer modes, simplifying the labeling process with automated suggestions.

• Row Labeling Extension — Introduce an intuitive shortcut setting to match shortcut keys in dropdown question types with the numeric label options.

LLM Labs

• Auto-generate Instructions — Improved the disclaimer copy message.

• Automated Evaluation — Improved the sorting mechanism in the automated evaluation result page.

• Deployment — Users can now define the retrieval scope in deployed applications.

• Direct Access LLMs — Added Claude 3.5 Haiku from Amazon Bedrock provider.

• Direct Access LLMs — Added Qwen model from Hugging Face provider.

• PII Masking — Improved the tooltip copy message.

• Sandbox — Users can now customize the context placement within system instructions and user instructions.

Bug fixes 🐞

NLP

• Fixed an issue in Conversational Labeling where rejected message labels were not visible after refreshing the page, despite the 'Show rejected labels in Review Mode' setting being enabled.

• Fixed an issue in Conversational Labeling where the speaker name would disappear after a reviewer resolved conflicting content.

• Fixed an issue where prompts in ML Assisted with LLM couldn’t be saved if the user navigated to another extension.

LLM Labs

• Fixed an issue where the Add application or Duplicate application button would trigger the creation of a new prompt.

6.87.0

November 07, 2024

What's new 💡

NLP

• Default project extension page — Updated the illustration image for Grammar Checker extension to enhance clarity.

• Document Labeling — Allow CSV files for Document Labeling projects.

• List of Files extension — Updated the popover copy when hovering over the labeler chip for documents with no assigned labeler to enhance clarity.

• Manage extensions dialog — Added a “Learn More” link for Bounding Box Labels extension to provide easier access to details.

LLM Labs

• Automated evaluation — Limited the number of multiple applications to five.

• Auto-generate instruction — Added a new feature to help users improve their system instructions.

• Knowledge base — Improved our add URLs functionality; users will now not need to add the protocol when adding URLs.

• Knowledge base — Improved the activity file dropdown to show the file source from external object storage.

• LLM Models — OpenAI models will now use PII masking.

• Sandbox — Hyperparameter configuration will now be retained when changing to a new model.

Bug fixes 🐞

NLP

• Fixed an issue in Conversational Labeling where the tooltip flickered when clicking the “+” button to apply the message label.

• Fixed an issue in Conversational Labeling that caused an error when creating projects with split files.

• Fixed an issue in Conversational Labeling where the message label was not highlighted when clicked from the Search extension.

• Fixed an issue in Span and Conversational Labeling where a different label in the same span was highlighted instead of the clicked label from the Search extension result.

LLM Labs

• Fixed an issue where the cost calculation is showing error when the cost is zero.

• Fixed an issue where the tooltip in sandbox hyperparameter is cropped.

6.86.0

October 31, 2024

What's new 💡

LLM Labs

• Direct access LLMs — Added new models: Claude 3.5 v2, Mistral Large 2, and Llama 3.2 via Amazon Bedrock providers.

• Knowledge base: External knowledge base — External knowledge base will now be hidden. Users will need to contact support to enable the external knowledge base feature.

• Knowledge base — Improved the empty state when the user applies a filter.

• Knowledge base — Users can now directly type the URL for adding a URLs file.

• Performance monitoring — Added a new feature to help users monitor the LLM application by regularly evaluating its performance over time.

Bug fixes 🐞

NLP

• Fixed an issue in Bounding Box Labeling where changing the zoom level during resizing or moving a bounding box would revert any changes made.

• Fixed an issue in Conversational Labeling where using project template or reusing project settings caused an error during project creation.

LLM Labs

• Fixed an issue where long file names or text in LLM Labs would overflow.

• Fixed an issue where the model's provider name would disappear every time the Sandbox was refreshed.

• Fixed a rubber banding issue in the hyperparameter configuration.

6.85.0

October 24, 2024

What's new 💡

NLP

• Bounding Box Labeling — Introducing a new and easier way to perform bounding box labeling: text selection. Users will be able to directly select text from native PDFs to create bounding boxes easily.

• Span Labeling:

  • Added a new 'None' option for line separators during project creation, ensuring no segmentation is applied during labeling.

  • Added a new text line spacing option in the personalization settings, allowing users to customize the spacing between lines for better readability.

  • Reduced the spacing in each line to provide a more compact view.

• Reset labeling work — A new capability to restart the labeling process from scratch without having to recreate the project in any labeling type projects.

• Row Labeling — Enhanced sorting for text questions to treat uppercase and lowercase letters equally, ensuring more accurate and consistent results.

• Projects Page — Added an initial width to the table to ensure consistent layout and improve readability when displaying projects.

• New Feature: ML-Assisted Labeling for Bounding Box labeling project

  • The Bounding Box project type now supports ML-assisted labeling with Custom API. This feature helps users quickly and efficiently label documents requiring bounding boxes, significantly speeding up the annotation process.

LLM Labs

• Added a new Anonymize PII feature — Users can now mask their prompt results in Sandbox.

• Added a new Use in Sandbox feature — Users can now directly use their created knowledge base in Sandbox with just one click.

• Evaluation — Users can now use applications from the saved application library when creating Evaluation project.

• Automated evaluation — Improved the copy message in the creation wizard.

• Models page — Improved the Models page UI for greater clarity.

• Onboarding tutorial — Improved the onboarding tutorial user experience. The backdrop will now only show for the first step of the onboarding tutorial.

Bug fixes 🐞

NLP

• Fixed an issue in the Add Documents wizard where the Preview step displayed the Span Labeling UI instead of the Conversational Labeling UI for Conversational projects.

• Fixed an issue where search results in Span and Conversational Labeling didn’t highlight text or labels while navigating across files.

• Fixed an issue in the Default Project Extensions page where the popover blinked when attempting to start the extension trial.

6.84.0

October 17, 2024

What's new 💡

NLP

• Bounding Box Labeling:

  • Added a new export format for LayoutLM.

  • Introduced the ability to merge multiple bounding boxes, allowing them to be treated as a single bounding box.

• Conversational Labeling:

  • Added a new field for the message label in the Datasaur Schema.

  • Displayed error messages when a user attempts to upload an unsupported file type in the Add Documents wizard.

• Grammar Checker Extension:

  • Updated the button name to improve clarity.

  • Updated the illustration image in the Manage Extension dialog to enhance clarity

• Prelabeled Document Project — Enhanced user experience by allowing users to exclude non-required questions.

LLM Labs

• Advanced Hyperparameters — Improved our Sandbox Advanced hyperparameters to allow null values.

• Automated evaluation — Automated evaluation now supports multiple applications. Users can now evaluate multiple applications at once.

• Direct Access LLMs — Added a new GPT-4o version from Azure OpenAI and OpenAI providers.

• Knowledge base — Improved the creation flow for external knowledge bases in our production. Users can now create external knowledge bases via the UI.

• Knowledge base — Increased the file size limit to 500 MB.

Bug fixes 🐞

NLP

• Fixed an issue in Bounding Box Labeling where an error message appeared when submitting bounding boxes without required attributes.

• Fixed an issue in Row Labeling where the slider value in the filter pop-up did not update when using the up or down arrows in the number field.

• Fixed an issue in Span Labeling where a selected span remained highlighted even when it was filtered out of the results, causing confusion as the span was no longer visible.

• Fixed an issue in Span Labeling where recently rejected labels didn't appear in the filtering results until the page was refreshed.

LLM Labs

• Fixed an issue where the models field in Sandbox was auto filled and produced an error when running the prompt after users disconnected their provider.

• Fixed an issue where the search box on the My Models page was unresponsive when users pasted a value.

Self Hosted Changes 💡

NLP

• IAM Role Support: The Datasaur Assist module now supports IAM roles, so users no longer need to specify access keys and secrets to run the module.

• Image Renaming: datasaur-dinamic is renamed to datasaur-assists. There should be no interruption to the existing service. However, if there are configurations that manually point to datasaur-dinamic-*, please update them to datasaur-assists-*.

6.83.0

October 10, 2024

What's new 💡

NLP

• Add Documents Wizard — Enhanced UI for Steps 1 and 2 to improve clarity and user experience.

• Bounding Box Labels Extension — Enhanced functionality to ensure that clicking on a label instance now jumps to the selected bounding box and opens the labelbox for improved navigation.

• Projects Page:

  • Added project owner information in the project table for better visibility.

  • Stored users' last configured sort order for the project list, allowing sorting by 'created date' or 'modified date' for better customization.

• Span Labeling — Improved the copy in the "Manage Functions" dialog for clearer guidance.

• Span Labeling — Improved the Search extension to display results based only on the filtered lines.

LLM Labs

• Direct Access LLMs — We have integrated Azure AI as a provider, including their Llama 3.1 model.

Bug fixes 🐞

NLP

• Fixed an issue in Conversational Labeling where message labels could not be replaced when clicking "Label all" in the Search extension.

• Fixed an issue in Document Labeling where right-clicking in the row area displayed an empty context menu.

• Fixed an issue in Row Labeling where right-clicking within a clickable row area did not display the appropriate context menu.

• Fixed an issue in the Search extension where clicking the "Label all" caused double labeling of the searched text.

• Fixed an issue where the project status remained read-only in Reviewer mode after adding a new assignee to the project.

LLM Labs

• Fixed an issue that caused the consumption of remaining free credits during failed evaluation project creation.

• Fixed an issue that prevented the programmatic API inference call from deducting the free trial credit.

• Fixed an issue where system information and user instructions were missing from the deployed API in Sandbox.

6.82.0

October 03, 2024

What's new 💡

NLP

• Add Documents Wizard — Added an error message when attempting to upload an empty file, providing clearer feedback.

• Analytics Extension: Easily track your labeler’s progress on ongoing projects by accessing relevant metrics. This new extension also includes document and team member filters for more customized calculation.

• Bounding Box Labeling — Introduced a new setting that automatically selects the bounding box when hovered over, allowing users to move or edit it without having to click on it first.

• Comment Extension — Added a loading indicator when the system is fetching comments to prevent user confusion about why nothing appears.

• Conversational Labeling — A new labeling type to support conversation-based use cases.

• Labels Extension — When “Allow marking unapplied label classes as N/A" setting is enabled, the instance counter with a value of 0 will be highlighted in red to indicate it is clickable.

LLM Labs

• Direct access LLMs — Added Llama 3.2 as our new model.

• Direct access LLMs — Improved our Gemini 1.5 series to support the newest version.

• Direct access LLMs — Improved our Hugging Face model streaming capabilities.

• Knowledge base — Revamped our knowledge base user flow, user will now can create the knowledge base and do embedding process easily.

Bug fixes 🐞

NLP

• Fixed an issue that prevented users from submitting incorrect arrow positions.

• Fixed an issue where Reviewers could add new options to dropdown attributes in Labelbox for Bounding Box Labeling.

• Fixed issues related to comments in Span Labeling projects:

  • The comment pop-up was covering the text when opening comments using the Comment extension.

  • Users were unable to open text comments from the Comment extension.

LLM Labs

• Fixed an issue where the empty state did not appear when a user opened an invalid URL.

• Fixed an issue where the system did not deduct API inference calls from the free trial quota of free trial users.

API Changes 💡

• Remove legacy API to retrieve daily usage.

Self Hosted Changes 💡

• Remove legacy usage tracking.

6.81.0

September 26, 2024

What's new 💡

NLP

• Custom Report Builder — Enhance date range handling when using the API by supporting multiple date formats, including Python datetime, for better compatibility.

• Comment Extension — Moved the comment list into the extension to make it easier to view and manage all project discussions in one place.

• Span Labeling — Added the ability to filter lines based on label status.

• Supervisor Role — Added access to their created project:

  • Apply and remove tag from an existing project.

  • Add documents to an existing project.

LLM Labs

• Added edit and delete functionality to our Custom models feature.

• Added a "Try in Sandbox" button to model cards for quick model testing in Sandbox.

• Added GPT-4 mini, GPT-1 preview, and GPT-1 mini as new Direct Access LLMs via Azure OpenAI.

• Added a snackbar notification when the user clicks the "Sync models" button on the Models page.

• Improved our Models page by using lazy loading to enhance browser performance.

• Improved the cursor icon in the Knowledge base list of files.

• Improved our Sandbox completion to use streaming capability.

Bug fixes 🐞

NLP

• Fixed an issue where marking a document as complete would jump to that document.

• Fixed an issue where draft answers in Document Labeling extension would disappear in mixed projects (Bounding Box + Document, or Span + Document) after applying labels.

LLM Labs

• Fixed an issue where the upgrade plan dialog was not showing when the remaining Knowledge base credits were exceeded.

• Fixed an issue where the upgrade plan button tooltip wouldn't show in the Knowledge Base.

• Fixed an issue where accessing an evaluation project would cause a blank screen.

• Fixed an issue where the error message was incomplete when adding a new model.

• Fixed an issue where the progress bar was not working during the file uploading process in the Knowledge Base.

6.80.0

September 19, 2024

What's new 💡

NLP

• A dialog will be shown to users when uploaded files exceed the maximum size limit during project creation or document addition, showing which files exceeded the limit and the maximum size allowed.

• Bounding Box Labeling

  • Directly able to move and edit a Bounding Box upon hover.

  • The copy shortcut now retains the latest edit of a bounding box, eliminating the need to save and re-select before copying.

• Extension — When full-height is applied for the extension style, enabling an extension from the Manage Extension dialog will now display it as the currently active extension, reducing confusion about its visibility.

• Row Labeling — Users can now resize table columns more easily, with expanded columns pushing other columns and allowing the table to adjust dynamically.

LLM Labs

• Added a Custom model feature as a new Model provider; users can now bring their own Models from their own endpoints.

• Added OpenAI o1 as a new Direct Access LLM.

• Added tooltips on the gearbox icon and three-dots icon in Sandbox.

• API calls from Sandbox deployment will now deduct free trial credits.

• Changed Vector stores term to Knowledge base.

• Improved Azure OpenAI and Vertex AI interfaces to conform to OpenAI standards.

• Improved our Knowledge base flow to allow users to immediately start embedding when clicking the Update button.

• Improved the loading state when creating knowledge bases, evaluations, and fine-tuning.

Bug fixes 🐞

LLM Labs

• Fixed an issue where the error message was not properly displayed when a user attempted to convert a PAYGO plan to an AWS Marketplace plan.

• Fixed an issue where the Hugging Face dedicated inference endpoint couldn't be connected to a Custom model feature.

• Fixed an issue where the Sandbox cost prediction caused infinite loading.

• Fixed an issue where users couldn't deploy and undeploy fine-tuned models.

6.79.0

September 12, 2024

What's new 💡

NLP

• ML Assisted Settings for Labeler — Reviewers can now enforce their ML Assisted settings, allowing Labelers to utilize the Reviewer’s ML Assisted settings. This ensures consistency in labeling through the Reviewer’s chosen configuration.

• ML Assisted with LLM Labs now supports Span-Based Project, expanding the range of project types that can benefit from this feature.

• PII Settings — Enhanced the ability by preventing user submitting empty field in the regular expression tab.

• Row Labeling — Added a new option to filter rows by “labeled rows” label status in Labeler mode.

• Supervisor Role — Added access to:

  • Save label / question set in a project to the library.

  • View and modify project settings.

• Search and Review Extensions — Added 'Part' information to the document name for projects created with split files to help differentiate them.

LLM Labs

• Added a sidebar to every main feature page.

• Added maximum and minimum values in the Automated evaluation report.

• Changed the term "Ground truth" to "Dataset."

• Released a new fine-tuning models feature.

• Standardized the Dataset column pairs across pages.

Bug fixes 🐞

NLP

• Resolved a problem where users were unable to undo predictions in ML Assisted after selecting "Accept All." The prediction list from the extension now correctly reappears.

LLM Labs

• Fixed an issue where clicking the triple dots icon button on the Vector stores list page would redirect to the Vector stores project page.

6.78.0

September 05, 2024

What's new 💡

NLP

• Activities Page — Added the ability to search for a project activities using the project ID.

• Added the ability to zoom in and out in documents using shortcut key for easier navigation.

• Enhanced the screen resolution warning and supported browser dialog for improved clarity.

• Export — Added the ability in Reviewer mode to include media files when performing a single file export.

• External Object Storage — Read only access for external ID on S3 to cover any security hole.

LLM Labs

• Added Huggingface as a new Direct access LLMs provider.

• Removed several deprecated LLMs from Models page.

Bug fixes 🐞

NLP

• Fixed an issue in the Search Extension where bulk labeling could not be applied to a span if part of that span had already been labeled.

• Fixed an issue where adding documents to an existing project was incorrectly recorded as 'Project Created' on the Activities page.

• Fixed an issue where the UI would freeze when editing a bounding box label with an empty value in a required dropdown.

LLM Labs

• Fixed an issue in the RAG Runner API Call runner.

6.77.0

August 29, 2024

What's new 💡

NLP

• Custom Report Builder — Labeling Table: A more granular team member segmentation (specifically reviewer) if total completed document is the only selected metric.

• Default Extension Setting — Enhanced the snack bar message when enabling an extension to provide more clarity.

• External Object Storage AWS S3 — Read-only access for external ID to improve security.

• Projects Page — Added the ability to search for project using the project ID.

• Project Analytics v2 — A better experience after a project is completed by sharing the relevant summary of the labeling result.

• Row Labeling — Added a new option to filter rows by “suggested label” label status.

LLM Labs

• Improved our Sandbox user interface by moving the model field out of the advanced settings.

Bug fixes 🐞

NLP

• Fixed an issue where numbers and content were slightly misaligned in Grammar Checker Extension.

• Fixed an issue where an error occurred when adding documents to a project via External Object Storage, ensuring users can only add documents using the same method as the initial upload during project creation.

• Fixed an issue where ML Assisted Labeling can't accept multiple answer in a dropdown question type.

LLM Labs

• Fixed an issue where an AWS Marketplace user signing up via Google Single Sign-On (SSO) would fail.

• Fixed an issue where opening the Sandbox resulted in a blank screen.

6.76.0

August 22, 2024

What's new 💡

NLP

• Enhanced the appearance of the PII settings dialog in Step 5 of the Project Creation Wizard.

• Improved the UI for handling empty Custom Report Builder report previews. If a configuration results in no data, the UI now clearly indicates that the data is indeed empty, preventing any confusion for users.

LLM Labs

• Added a new export format in Rating evaluation to support AWS Bedrock fine-tuning.

• Added evaluator and embedding model information in the automated evaluation summary table.

• Implemented collapse and uncollapse buttons in Sandbox for better navigation.

• Revamped Models page to support fine-tuning models.

Bug fixes 🐞

LLM Labs

• Fixed an issue where a user can't rename vector stores due to failed customer plan.

6.75.0

August 15, 2024

What's new 💡

NLP

• Added a confirmation dialog when closing project settings to prevent accidental loss of changes.

• Enhanced the appearance of the Select from Library dialog for Label and Question Set in Step 3 of the Project Creation Wizard.

• LLM-Assisted Labeling is now available for Span-Based projects.

• Review Sampling:

  • Add validation to prevent users from marking a project as complete if the review sampling threshold has not been met.

  • Now supports mixed labeling for both Span and Document Labeling projects.

• Row Labeling — UI improvements for the comment indicator in the table for better and clearer information display.

• Row Labeling — Enhanced scrolling behavior for multiple selection to improve user experience.

• Supervisor Role — Added access to the Label management page.

LLM Labs

• Added an informational dialog when users delete an evaluation project.

• Added AWS Bedrock as our new model provider.

• Added an informational dialog when users delete an evaluation project.

• Added Deepeval as our new evaluator in the automated evaluation feature.

• Improved our SageMaker JumpStart provider integration to use the TGI Message API.

• Improved our Sandbox application configuration to include three decimal places for the hyperparameter.

• Improved the Sandbox application's throttling mechanism.

• Updated the Datasaur logo.

Bug fixes 🐞

NLP

• Fixed a bug where the Labeling Function Analysis did not appear after clicking "Predict Labels" in Data Programming for Span-Based projects.

• Fixed an issue where the document name field was incorrectly displayed in the ML Assisted Custom Provider example.

• Resolved a problem where Datasaur Dynamic AWS Sagemaker data was not showing properly.

LLM Labs

• Fixed an issue where the delete button tooltip was showing an incorrect message.

6.74.0

August 08, 2024

What's new 💡

NLP

• Bounding Box Labeling — Improved the positioning of label boxes for a better accuracy and ease of use.

LLM Labs

• Added a banner and a quick tutorial for Llama 3.1, the new model from LLM Labs.

• Added a new feature that allows users to filter usage based on the Feature Type and Model utilized.

• Added a new feature that allows users to generate an API key from the Deployment page.

• Changed the product module term from "Playground" to "Sandbox".

• Improved and Enhanced LLM Labs Onboarding Tutorial.

• Improved the maximum output for Mistral 7B Instruct v3 in the AWS SageMaker provider.

• Improved the mechanism for fetching LLM models in the Sandbox.

• The name of the LLM Playground extension was changed to LLM Sandbox in the Rating Evaluation feature.

Bug fixes 🐞

NLP

• Fixed an issue where Bounding Box Labeling project turns blank when editing label set in JSON tab.

LLM Labs

• Fixed an issue where opening the account settings page would cause the page to crash.

• Fixed an issue where redeploying the Sandbox would cause a blank page.

• Fixed an issue where the copywriting in the onboarding tutorial was incorrect.

• Fixed an issue where the pop-up dialog was not showing when users exceeded their quota limit during the free trial.

• Fixed an issue where the width of the code block completion result was incorrectly displayed.

• Fixed an issue where uploading large text files in LLM Labs caused an out-of-memory error.

6.73.0

August 01, 2024

What's new 💡

NLP

• Added an export information event to the Activities page for improved tracking and monitoring of activities.

• Bounding Box Labeling — Adjustments to when the bounding box tooltip is shown for a better user experience

• Bounding Box Labeling — Improved export results to ensure bounding box coordinates stay within image boundaries.

• Custom Report Builder: The report preview now uses real data instead of the previously used dummy data.

• Enhanced the appearance of the Advanced Assignment dialog in Step 4 of the Project Creation Wizard.

• Export — Added the ability in Reviewer mode to include media files when exporting all files.

• LLM-Assisted Labeling: Datasaur now supports more Large Language Models for your labeling journey! Utilize OpenAI, Anthropic Gemini, and Cohere to enhance your workflow.

• Row Labeling — UI improvements for the conflict indicator in the table for better and clearer information display.

LLM Labs

• Added a new snackbar to notify users when the embedding process take a long time to finish.

• Added an error message when users have no billing plan.

• Added Llama 3.1 as one of our new Direct Access LLMs.

• Improved our onboarding tutorial to use a spotlight backdrop.

• Labeler and reviewer roles now can create their own LLM playground.

Bug fixes 🐞

NLP

• Fixed an issue in Bounding Box Labeling project where the question default value was not exported while using the Datasaur Schema format.

• Fixed an issue where excluding the date segment on the Progress group table in the Custom Report Builder resulted in an empty report for Bounding Box Labeling.

LLM Labs

• Fixed an issue where the spacing in the LLM application was incorrect.

• Fixed an issue where vertical scrolling was not visible on the evaluation project creation page.

6.72.0

July 25, 2024

What's new 💡

NLP

• Bounding Box Labeling — Adjust the bounding box style to create a more intuitive interface.

• Default Project Extension Setting — A new workspace-level setting to customize default extensions enabled in every project created for each project role.

• Improved the tooltip messaging in the Projects, Label management, and Members page for better understanding.

• Row Labeling — UI improvements for the table header and toolbar to create a more intuitive interface.

LLM Labs

• Added a new "Pay as you go" UI flow for users who signed up through AWS Marketplace.

• Added GPT-4 mini as a new Direct Access LLM.

• Added "Pay as you go" and subscription payment plan features in LLM Labs.

• Added "Processing" visual feedback after users click "Run Prompt" in LLM Labs Playground.

• Added remaining free trial credits information on the Evaluation page.

• Disabled the Azure OpenAI embedding models in LLM Labs.

• Improved the "Pay as you go" plan alert message after users add their payment details.

Bug fixes 🐞

LLM Labs

• Fixed an issue where remaining free trial credits were still showing for non-free trial users in the LLM playground.

• Fixed an issue where remaining free trial credits were still showing for non-free trial users in Vector stores.

• Fixed an issue where the column sequence order in LLM Evaluation: Rating and Ranking was incorrect.

• Fixed an issue where the total cost differed between usage diagrams and the totals dialog.

• Fixed an issue where users could not load credentials from any providers.

• Fixed an issue where users could still save their payment details without adding their credentials.

6.71.0

July 18, 2024

What's new 💡

NLP

• Added information about reserved project tags in step 1 of the Project Creation Wizard to prevent confusion when users attempt to add tags that they have previously added but are not appearing.

• Improved the messaging when the filters displays zero lines in the Row Labeling project for better clarity.

• Review Sampling — Enhanced the span counting logic to increment only after all labels in a span have been reviewed.

LLM Labs

• Added AWS Bedrock embedding models to our Vector stores.

• Added a Free Trial feature in LLM Labs. Users will receive prompt and embedding quotas during the Free Trial process.

• Added user prompting activities to our Usage page.

• Enabled OpenAI GPT series models as our Direct Access LLMs.

Bug fixes 🐞

LLM Labs

• Fixed an issue where the LLM playground would become stuck when users ran a prompt and refreshed the page before it finished processing.

• Fixed an issue with calculating the average score in the LLM Evaluation evaluator.

6.70.0

July 11, 2024

What's new 💡

NLP

• Analytics Overview — Display the last updated information for metrics on the Home tab.

• Enhanced the relabeling behavior for rejected labels in Span Labeling projects during Reviewer mode when 'Enable checkboxes in labelbox' is checked.

• External Object Storage — Enhance protection for Google Cloud Storage by introducing a new security token field. Rest assured, your existing buckets will work as usual.

• Improved the Grammar Checker extension to prevent cropped content from appearing.

• Introduced a new access to configure Review Sampling in project settings, enhancing the user experience by allowing direct activation within projects.

• Project Analytics (project details page) — Display the last updated information for metrics on the Trends tab.

• SCIM — Support for the new Supervisor role.

LLM Labs

• Added a feature to handle Vertex AI model safety configuration attributes. Users will now be prevented from asking offensive, insensitive, or factually incorrect questions.

• Added a new banner to showcase Claude 3.5 integration with LLM Labs.

• Added a new export feature in automated evaluation.

• Improved the automated evaluation result detail dialog.

• Improved the automated evaluation UI flow.

• Improved the color coding of automated evaluation results in the automated evaluation details page.

• Improved the custom delimiter detector to accept semicolon-delimited CSV files in automated evaluation.

• Limited the number of prompts and applications in a single playground. Users can now add only 5 applications and 20 prompts in a playground.

Bug fixes 🐞

NLP

• Fixed an issue where bounding boxes remained gray after removing labels in Bounding Box Labeling projects.

• Fixed an issue where the go-to-line navigation reset, causing users to start from the beginning of the row in Row Labeling projects.

LLM Labs

• Fixed a bug in the title transition between the "Run all" and "Run selected" buttons.

• Fixed an issue where the cost calculation was not accurately calculating costs for evaluations using the Direct Access LLMs from OpenAI.

6.69.0

July 04, 2024

What's new 💡

NLP

• Added supported file types information in Step 1 of the Project Creation Wizard for Bounding Box Labeling: YOLO, LabelMe, and Pascal VOC.

• Adjust the dialog and snack bar messages for creating and editing Bounding Boxes to ensure clearer information.

• Custom Report Builder: A new group table called Progress. It can be used to track your labeling work which contains these metrics: Total Documents Completed, Total Documents Answered, Total Rows Answered, and Total Sentences Labeled.

• Improve the appearance of the delete button on the Label Management page to avoid cropping issues.

• Multiple Target Texts and Target question for ML Assisted - LLM Labs. Users can now provide more context to their LLM application in LLM Labs and receive multiple predictions in JSON format. These predictions can be sent to the NLP Platform using ML Assisted.

• Rename the question-answering extension for Row Labeling project. Previously, it was called Document labeling extension.

• Switching from flat comments to threaded comments in the editor for Span Labeling, with replies automatically expanded by default to enhance the user experience.

LLM Labs

• Added a new automated evaluation feature.

• Added a new feature to let users choose from ground truth when selecting datasets in the automated evaluation creation wizard.

• Added a new error message for when there is an error creating a vector store.

• Added a new copy button feature in LLM Playground when generating code as a completion result.

• Added OpenAI embedding models as direct access models.

• Added GPT-4 from OpenAI provider as a new Direct Access LLMs.

• Equalized the display of vector store URL documents based on their paths.

• Improved checkbox consistency in the LLM playground after page refresh.

• Improved the file preview for .docx files in Vector Store to prevent cropping.

• Improved the behavior when users click the "retry failed vector store" button; the user page will stay the same and not be redirected.

• Users will be redirected to the correct module they are currently working on.

Bug fixes 🐞

NLP

• A more accurate calculation of Total Time Spent by excluding the idle time of each user.

• Fixed an issue in OCR projects (Span Labeling with documents as media files) where documents were displayed cropped when resized.

• Fixed an issue where the context menu was still shown for rejected labels even when the 'Show Rejected Labels in Review Mode' option was enabled.

• Fixed the display in the reply box in comments to ensure resolved comments are displayed correctly and do not appear cropped.

• Fixed an issue in Step 3 of the Project Creation Wizard where the tooltip incorrectly listed file types other than .CSV for LLM Ranking and LLM Evaluation projects.

• Fixed the answers not updated in Document Labeling extension on Predictive Labeling.

• Improving ML Assisted with LLM Labs Error Message.

LLM Labs

• Fixed an issue where the evaluation list table displayed no results when a manual evaluation record was present.

• Fixed an issue where LLM Evaluation returned a blank response after undeploying an application.

6.68.0

June 27, 2024

What's new 💡

NLP

• A confirmation pop-up message is added when deleting comments to prevent users from accidentally deleting them.

• Introducing a new setting in Step 4 of the Project Creation Wizard for review sampling capabilities.

• Project settings can now be accessed directly from the Projects page by clicking on the three-dot icon of the project, providing faster access to the settings.

• The 'Add Documents to an Ongoing Project' feature is now enabled for all team workspaces.

LLM Labs

• Added a new Automated Evaluation feature in LLM Labs.

• Enhance the main page of LLM Labs Playground by correcting minor grammatical errors. Introduce an empty state for instances where no search results are found in the Playgrounds section, and similarly, add an empty state for when there are no results from applied filters.

• Improve the Model Page UX by adding a helpful message when no search results are found. A success banner should appear on the Manage Providers tab after successfully connecting a model. Add tooltips to provide clear and concise explanations for the displayed information.

• Improve LLM Evaluation Pagination Filter, Status, and Project Kind to Support a new LLM Labs Automated Evaluation feature.

• Remove the Expand Model Icon in LLM Playground Application Area.

• We've changed the dialog box message that appears when a user deletes files in the vector store to deleting files.

• Update OpenAI Model Provider Key Instructions and Correct Grammar Mistakes.

• We've changed the font color for processing time in the completion result to black.

• Integrate Claude 3.5 Sonnet into Amazon Bedrock Direct Access LLMs.

Bug fixes 🐞

NLP

• Fixed an issue where submitting using the Enter key reset the bounding box position and/or vertices to their original positions.

LLM Labs

• Fixed an issue where delete button in the Playgrounds main page and the vector store table appear multiple times.

• Fixed an issue where the days filter was not working if sorting was implemented on the Usage page.

API changes 💡

Extension

• We have separated the question-answering extensions for Row labeling and Document labeling projects. This change should not affect any existing projects.

Project creation

• Due to the changes above, there are side effects to the project creation API. For users using our API to create projects programmatically, please check your JSON payload file, especially if you are creating Row labeling projects.

  • How to adjust:

    • If your payload includes the kinds ROW_BASED, remove any mention of DOCUMENT_LABELING_EXTENSION_ID, or replace them with ROW_LABELING_EXTENSION_ID.

    • Otherwise, you should be safe to reuse your existing JSON file as usual.

6.67.0

June 20, 2024

What's new 💡

NLP

• Update the "Save Configuration" mechanism to "on change" for ML Assisted Labeling fields in the Row Based Project.

• Workspace admins now have access to a new GraphQL endpoint. This endpoint allows you to retrieve labels marked as N/A in a project, providing a more streamlined and efficient way to manage your labeling data.

LLM Labs

• Activity Vector stores now display new features that highlight deleted files.

• Added new tooltips to buttons in Vector stores.

• A new feature to allow users to update Base URL, username, and password in the External Vector Store.

• Improved the readability of the onboarding tooltips to reduce user drop-off.

Bug fixes 🐞

NLP

• Fixed an issue where the project settings edit indicator in the Administrator tab is not cleared after changes in the tab are reset.

• Handle the long text on Analytics pages properly.

LLM Labs

• Fixed an issue where the Application name in the snack bar and dialog was showing extra whitespace.

• Fixed an issue where a user could submit an invalid file in LLM Labs manual evaluation.

6.66.0

June 13, 2024

What's new 💡

NLP

• Custom Report Builder — Use two decimal places instead of round number for time spent metrics.

• Made adjustments to ensure that Metadata Extensions will only be added to Span and Row Labeling projects.

• View the Analytics Overview filtered by project tag.

LLM Labs

• Vertex AI Embedding Models are now available for use with Vector Stores.

• Users now able to rearrange the order of applications in the LLM Playground interface.

• Improve the input option for hyperparameters configurations in LLM Playground Application.

• A new progress bar now indicates the loading process when opening a newly added vector store with many files.

• A new indicator in LLM Playground flags prompt results that might be outdated due to recent model or data updates.

• A new Application Saving feature. This will allow users to save and use their Application across workspace.

• The Vector Stores page now displays the total number of files being processed and the number successfully processed.

• Added the BaseURL for external vector stores to the Vector Stores page. Now you can easily view the base URL for each store directly within the page.

Bug fixes 🐞

NLP

• Fixed an issue in Bounding Box Labeling project where the label box would close immediately after pressing the Esc key.

• Fixed an issue where accessing the Label Error Detection extension becomes stuck on loading when using the Supervisor role.

• Fixed an issue where moving or editing the bounding box label would reset the latest caption and/or attributes.

• Fixed the missing efficiency chart on the member page.

LLM Labs

• Fixed an issue where a snack bar error message appeared after a user deleted a file in Vector Stores.

• Fixed an issue where the prompt card disappear while typing on the prompt input.

6.65.0

June 06, 2024

What's new 💡

NLP

• A new export format in Span Labeling: Comma-separated values (CSV).

• Enhance project activity tracking by displaying information regarding deleted projects on the Activities page.

• Metadata extension is enabled to all project types to accommodate users importing data through Datasaur Schema.

• Enhance user experience by setting the default project extension style to Full-height to avoid narrow extensions. This change applies to:

  • Span and Bounding Box Labeling project in both Labeler and Reviewer mode.

  • Row and Document Labeling project in Labeler mode.

• Additional field named Target text in all ML Assisted Labelling Row based providers. This field will act as input data for ML Assisted prediction. Previously, all columns were counted as input data, but now you can select which columns are relevant.

• New Data Programming initial template for Row and Span based project.

LLM Labs

• Implement support for longer context windows in LLM Labs. This will allow users to control the maximum context window size more explicitly.

• Allow filtering activities in Vector Store based on files uploaded through external object storage.

• Added a new progress dialog when updating the vector store.

• Decreased the line spacing between prompts in LLM Labs Playground completions.

• The LLM Playground window now offers a more balanced view by default, with a 3:5 aspect ratio for both application and prompt sections.

• Integrate new Amazon Bedrock LLMs, including Mixtral, Mistral, Claude, Cohere, and Amazon Titan variants.

• The Knowledge Base and Document Viewer in the Vector Store now have a 1:2 aspect ratio by default.

• Increased the character limit for prompts in the LLM Playground.

• The Vector Store main page will now display additional information, including embedding model, chunk size, overlap percentage, and the total number of files.

• The External Vector Store URL is now shown on the Vector Store page.

Bug fixes 🐞

NLP

• A better empty state IAA explanation for Bounding Box Labeling.

• Fixed an issue in the Activities Page where error messages were not displayed when encountering errors.

• Fixed an issue in the Bounding Box Labeling project where a dialog box prompting for an update to the new version appeared when editing label set via the JSON tab.

• Fixed an issue in the Project Creation Wizard where the preview kept blinking and displayed incorrectly when uploading media file with Datasaur Schema (JSON).

• Fixed an issue with a component when hovering over it, and corrected the tooltip in Label Error Detection.

LLM Labs

• Fixed an issue in the LLM Playground where models weren't returning the corresponding output chunks.

• Fixed an issue in Vector Store where the progress dialog text and bar didn't automatically close.

• Fixed an issue in Cost Estimation Calculation in LLM Playground where the cost was inflated.

6.64.0

May 30, 2024

What's new 💡

NLP

• A new workspace role: Supervisor — This role is responsible for managing labelers and reviewers.

• Project creation payload has been adjusted to allow the use of email address instead of teamMemberId.

• Users will be able to include conflicted labels and answers in the export result.

• Updated the supported file types for Span Labeling with audio and document as media files in Project Creation Wizard.

• Users now have their mapping results for synchronized scrolling feature to be saved for every user assigned to a document.

  • This means only one assignee needs to wait for the mapping process to complete.

LLM Labs

• Users now will have the ability to resize the LLM Labs Vector Store Knowledge Base and File Properties sidebar, allowing users to adjust its width as needed to view the added files more easily.

• Users now will have the ability to resize the Application and Prompt section in LLM Playground.

• Users now can “Run” the selected LLM Playground prompt in parallel, allowing to run other prompt without waiting the current prompt is finished.

• We’ve upgraded the integration to the newer version of Azure Open AI API, so the system should be able to retrieve the deployed model's version information from Azure Open AI.

Bug fixes 🐞

NLP

• Fixed inaccurate calculation on Custom Report Builder when changing labels, accepting labels on a different day than it’s applied.

• Fixed truncated data when generating the Custom Report Builder. It can now support up to 100k rows.

LLM Labs

• Fixed an issue where there’s mismatch between server data, so the response should be what expected from user request.

• Fixed an issue where the “Run” prompt behavior have a different value.

• Fixed an issue where the LLM Playground onboarding tooltip is not showing.

• Fixed an issue where user can’t create an External Vector Store.

• Fixed a Typeorm Migration Error in LLM Vector Store embedding process.

• Fixed an issue where the triple dot menu for Amazon Bedrock models is not showing in my models page.

• Fixed an issue where the LLM Vector Store status changed to “In Progress” when user updates the Vector Store name.

6.63.0

May 22, 2024

What's new 💡

NLP

• New Design for list of label errors in Label Error Detection Extension.

• Pascal VOC format is now supported for both import and export in Bounding Box Labeling project.

• Users can now paste each bounding box multiple times.

• Users can now select multiple bounding boxes using the Ctrl key, enabling them to copy, paste, and delete those selected bounding boxes all at once.

• Users can now upload multiple images with different file types during Project Creation.

LLM Labs

• Added GPT-4o, GPT-4 Turbo, Meta Llama 2, Meta Llama 3, and Gemini as new Direct Access LLM models in LLM Labs.

• Introduced a Knowledge Base Reference feature in LLM Playground to help users compare each chunk with the completion result.

• Added a feature in LLM Labs to edit the LLM Playground Application name.

• Improved the UI of the LLM Labs Models and Vector Store pages.

• Enhanced LLM Playground performance when adding new Applications and Prompts.

Bug fixes 🐞

• Fixed an issue where multiple pinned cell metadata were not displayed.

6.62.0

May 16, 2024

What's new 💡

• Support YOLO and LabelMe XML formats on both import and export for Bounding Box Labeling project.

• A better validation message in step 2 of Add Documents Wizard to handle the maximum limit of total assignees in a project (100 assignees, including existing and new) while adding assignees.

Bug fixes 🐞

• Fixed an issue where no email was sent to the user when exporting a project or file using S3.

• Fixed an issue where questions duplicated when adding a JSON/JSONL document in Add Documents Wizard before opening the newly created project.

• Fixed an issue where the arrow label with status LABELED remained in the review document result and didn't display in the UI.

6.61.0

May 08, 2024

What's new 💡

• Before, Datasaur will save after a single edit, either resizing, or moving the bounding box. Now, users can make multiple consecutive edits without having to wait for the saving process. To save the edits, simply click outside the label box area.

• Replace the DistilBERT OPIEC model provider with the newer FewNERD model for the NER tagging project.

Bug fixes 🐞

• Fixed several bugs in Label Error Detection.

6.60.0

May 02, 2024

What's new 💡

• We have made several improvements to the Bounding Box Labeling project. Users can now:

  • Edit and drag a single vertex of the bounding box.

  • Copy and paste a bounding box using keyboard shortcuts and the context menu.

  • Delete a bounding box using the Delete key.

Bug fixes 🐞

• Fixed an issue where enabling markdown parsing in the Row Labeling Project prevented thumbnails or large media from showing the image preview in Step 2 of the Project Creation Wizard.

• Fixed an issue where ML Assisted Labeling for Row Based can't predict all rows in Reviewer Mode.

• Fixed an issue where snack bars did not automatically close when multiple were opened in a short period of time.

• Fixed an issue where the colored dot in the status column wasn’t aligned at the top in the Projects page.

• Fixed an issue where the date segmentation was not selected, leading to inaccuracies in the Custom Report Builder (CRB) calculation.

6.59.0

April 25, 2024

What's new 💡

• Users can now edit their comments in Span and Row Labeling Projects.

• Introducing Synchronized Scrolling capabilities in the OCR Span Labeling Project, allowing users to view the Text Editor and Document Viewer synchronously.

• Adding the ability to automatically jump to the next document after marking a document as complete.

  • Users can enable this through the Project Settings > Personalization tab.

• Launching the Label Error Detection for beta access. This will significantly accelerate your review process by recommending the reviewers to check labels with highest error probabilities! You can give it a try by contacting our support team at [e-mail].

6.58.0

April 18, 2024

What's new 💡

• Users can now add project tags through Project Creation Wizard Step 1.

• The List of Files extension now contains the number of labelers who have or have not marked the document as complete.

Bug fixes 🐞

• Fixed an issue where there is a different trash icon color between the existing option and the question in the Edit question set dialog.

6.57.0

April 04, 2024

What's new 💡

• Labelers can now transfer their work by using transfer assignment ownership feature.

Bug fixes 🐞

• Fixed an issue where renaming and reordering questions didn't function properly.

• Fixed an issue where users were unable to move to another token after selecting one in a different document while using the Search extension.

• Fixed an issue where the Congratulations dialog didn’t appear when enabling the "Allow marking unapplied label classes as N/A" setting.

• Fixed an issue where the number of conflicting answers did not match the number of conflicted lines in the Review extension for the Row Labeling project.

6.56.0

March 28, 2024

What's new 💡

• Users can now easily adding one or multiple documents to their ongoing labeling project.

• We’ve enhanced the labeling experience by allowing users to mark labels that are not applied to a document as N/A (not applicable) in the Labels extension.

• We’ve enhanced the reviewer capabilities in Span Labeling project by adding ability to accept or reject all conflicted labels.

• We’ve made some UI improvements to the Assignment tab in Project Settings menu.

Bug fixes 🐞

• Fixed an issue where there was a mismatch wording and the sample template text will be covered by the options in File Transformer page.

• Fixed an issue where the highlighted area are overlapped when searching an option in hierarchical dropdown question.

6.55.0

March 21, 2024

What's new 💡

• We've improvement for Datasaur Dinamic: Add Input Columns where user can choose single or multiple columns as their training input context.

Bug fixes 🐞

• Fixed an issue where user redirects to team workspace not working properly.

6.54.0

March 14, 2024

What's new 💡

• We’ve enhanced the user experience by displaying total item information to all tables.

• We’ve improved the error messages for better clarity when user selecting the wrong project kind.

• We’ve introduced a brand new user package called “Starter” with all the details included.

Bug fixes 🐞

• Fixed an issue where the free trial banner looked cramped with multi-line text.

• Fixed an issue where users encountered error when creating document-based project with PDF files from External Object Storage.

6.53.0

March 07, 2024

What's new 💡

• Users can now collapse grouped attribute question type for a cleaner and easier navigation experience.

• We have enhanced the notification popover for project creation notifications.

Bug fixes 🐞

• Fixed an issue where the loading message didn’t appear when users rotated multi-page files in Document Labeling projects.

6.52.0

February 29, 2024

What's new 💡

• We've added project’s ‘completedAt’ information to the exported project results for better tracking.

• We’ve enhanced our question calculation capabilities to ensure smooth performance, even with large dynamic question sets.

• We've enhanced the ‘pages’ field within the Datasaur JSON Schema by eliminating unnecessary fields, leading to improved data structure clarity.

Bug fixes 🐞

• Fixed an issue in the Bounding Box Labeling project where multiple images were displayed as blank in Chrome version 121.

• Fixed an issue where ML Assisted with Open AI provider can't provide answer with certain symbols.

• Fixed an issue where question logic fields had incorrect initial values in Question Logic dialog.

• Fixed an issue where the Activities page displayed the user who created the project instead of the user who marked the project as complete.

6.51.0

February 22, 2024

What's new 💡

• We’ve enhanced the appearance of all the links.

Bug fixes 🐞

• Fixed an issue where there were missing gaps display in the OAuth Credentials form.

• Fixed an issue where there was a mismatched position when using whitespace tokenizer and char selection setting in Span Labeling projects.

• Fixed an issue where the error message kept blinking when removing the file keys in Project Creation Wizard Step 1.

• Fixed an issue where users encountered errors when creating a project while rearranging the file keys from External Object Storage.

• Fixed an issue where users wanted to reuse project settings, but there was a mismatch in the selected fields for Personally Identifiable Information (PII) in Project Creation Wizard Step 5.

• Fixed an issue where there was incorrect redirection for the 'jsonl' supported format link in Project Creation Wizard Step 1.

6.50.0

February 15, 2024

Bug fixes 🐞

• Fixed an issue where users experienced inconsistent behavior while uploading question set in the Project Creation Wizard, Label Management, and the Document Labeling extension.

• Fixed issues related to exporting projects using the triple dots on the Active Projects page.

  • Unable to rename the exported result.

  • Unable to open the Format and Delivery Method dropdown by clicking the text.

6.49.0

February 07, 2024

What's new 💡

• Added an information banner and disabled LLM Lab ML Assisted form when there is no application deployed in LLM Labs.

• Enhanced the performance when users open dropdown question type with numerous options in Row or Document labeling project.

Bug fixes 🐞

• Fixed an issue where page dimensions for image/PDF files were not included when users exported projects using Datasaur schema or file transformer.

• Fixed an issue where arrow labels were not completely replaced when users attempted to change old labels to new ones.

• Fixed the unexpected calculation in the quality chart.

6.48.0

February 01, 2024

What's new 💡

• We've enhanced our Span Labeling project by introducing the capability to adjust the span selection for labeled span.

• We've enhanced the Custom Report Builder with a direct download feature. Now, you can easily access your report using the direct download option.

• We’ve enhanced the error message to improve the user experience when creating the project with CoNLL-U format.

• We’ve polished the email copy for inviting a member during workspace joining to make it more general.

Bug fixes 🐞

• Fixed analytics chart calculation for LLM Evaluation projects.

• Fixed an issue where the header and columns in Project Creation Wizard Step 2 wouldn't reset after uploading new files.

• Fixed an issue where clicking on a comment in the Span-labeling project would cause the view to jump to the top of the sentence.

6.47.0

January 25, 2024

What's new 💡

• We’ve added the ability for users to select which rows they want to be automatically answered with the ML Assisted Extension in Row Labeling projects.

• We’ve enhanced the user experience in Document and Bounding Box labeling projects by adding tooltips and ellipses to long-option dropdown questions.

• We've released Datasaur Dinamic using Hugging Face AutoTrain for Span Labeling project.

Bug fixes 🐞

• Fixed an issue where users encountered errors with a minimum of 0 or -1 number of parts during file splitting in Project Creation Wizard with input files from External Object Storage.

• Fixed an issue where users received an error "Extension not found" when attempting to repeatedly enable all extensions.

• Fixed an issue where users were unable to view the full content of an image or file after rotating it in Bounding Box labeling projects.

• Rename LLM Lab to LLM Labs in ML Assisted Extension provider.

6.46.0

January 18, 2024

What's new 💡

• We’ve add a unique constraint for the label class name. As a result, we've initiated a migration process for existing label sets containing duplicate label class names.

• We’ve made some improvements to the validation checks for label set editing in Bounding Box labeling projects.

• We’ve made some improvements to the Active and Archived Project table to create a more intuitive interface.

• We’ve polished the Manage tags dialog to a new design for a more intuitive user experience.

Bug fixes 🐞

• Fixed an issue where incorrect error messages were shown during the creation or modifying a label with an invalid color in label set management.

• Fixed an issue where users could change the question type to a type other than dropdown in Row labeling for tabular embedded view.

• Fixed an issue where users encountered an error while trying to access the Document Labeling extension with saved answers in draft.

• Fixed issues with ML Assisted and Data Programming Target Questions or Label Set names being too long.

• The webhook request should work as expected now, as there will be no duplication anymore.

6.45.0

January 11, 2024

What's new 💡

• Newer and better looks on External Object Storage page.

• SCIM 2.0 integration — You can easily provision and sync users in a Workspace with your preferred Identity Provider. Currently, we officially support the integration with Okta.

• We've included a video link, showcasing our feature in the gamification box.

• We've modified several onboarding checklists to help users navigate our app more easily.

Bug fixes 🐞

• Always show the number of file progress in Review Extension.

• Fixed a bug on ML Assisted Accept and Reject all button.

• Fixed an issues in Reviewer mode, when a user resolves a conflicting answer and then undoes it, it does not revert to the conflicting answer.

• Improve Data Programming error message.

6.44.0

January 04, 2024

What's new 💡

• Added tooltips for dropdown and text with multiple-answer question types to provide a complete display of the text.

• Adjusted onboarding projects to better support ML-assisted labeling feature.

• The stable version of Custom Report Builder, elevating the user experience with a refreshed UI design. Additionally, introducing the preview capability, allowing users to see the generated data and gain a visual insight into the report. It is important to note that the preview feature uses dummy data and the information displayed does not represent actual data.

• Enhanced the Datasaur schema export format by introducing the Document ID field.

• Improved error messages for reviewers when modifying ‘Question set modification’ or ‘Label set modification’ settings.

Bug fixes 🐞

• Fixed an issue where the label set count in the library was displayed incorrectly in the Labels extension when users wanted to replace an existing label set.

• Fixed an issue where there was no text gap between the last label class of a label set and the name of the next label set in the ‘Labels not applied’ dialog.

• Fixed an issue where the file progress for each member in the Review extension did not update to the latest number until refreshed, when reviewers made changes in Advanced Assignment settings.

6.43.0

December 21, 2023

What's new 💡

• We’ve added a capability to add new questions to an existing project question set via Document Labeling extension's edit question set.

• We’ve added a loading dialog when users apply label via the “Label All” button in Search extension.

• We’ve added image rotation capability on OCR, Bounding Box, and Document Labeling project.

• We’ve made some improvements to the File Transformer to create a more intuitive interface.

• We’ve now support the native JSONL format in the Row Labeling project.

Bug fixes 🐞

• We’ve fixed a bug where the “Apply Label” button appeared even when no label was selected.

• We’ve fixed a bug where the "Next" and "Launch Project" buttons appeared cut off when it is loading in Project Creation Wizard.

• We’ve fixed a tooltip error on timestamp’s text field when playing audio files in Audio player.

• We’ve fixed a truncated message related to Datasaur maintenance.

6.42.0

December 14, 2023

What's new 💡

• We have implemented multiple select in team member removal.

• We have launched pinned project template to enhance the user experience. This feature is currently available only in the Datasaur-hosted version.

Bug fixes 🐞

• Fixed an issue where user needed to press tab twice from password to the organization field in the sign-up form.

• Fixed an issue where error message for column number validation for excel files wasn’t displayed in Project Creation Wizard Step 1.

• Fixed an issue where the long file transformer name didn't display properly in Project Creation Wizard Step 1.

• Fixed an issue where the error message was not displayed when file transformer name exceeded the character limitation.

• Fixed an issue where the long script code was cut-off in the file transformer editor.

• Fixed an issue where there was empty space in the Go menu when user opened the bounding box labeling projects.

• Fixed an issue where Undo and Redo were still enabled after marking Row Labeling project as complete.

• Fixed an issue where an error message did not appear when uploading incorrect file types for Question Set.

6.41.0

December 07, 2023

What's new 💡

• We’ve added "Learn more" links in each section to assist users in gaining a better understanding.

• We’ve added information about the timestamp of when a label or answer was annotated to our GraphQL Schema.

• We’ve made some improvements to the UI tab components to create a more intuitive interface.

• We've integrated Datasaur ML Assisted with Amazon SageMaker Provider. This enables you to directly utilize your model in Amazon SageMaker and use ML Assisted to assist with your labeling in Datasaur.

• We've added the Evaluation Metrics. This enables you to evaluate the Accuracy, Precision, Recall and F1 score of your labeling process. This feature applies for Row Labeling projects with dropdown, hierarchical dropdown, checkbox, and radio button questions (excluding multiple answers) after you have completed the project.

Bug fixes 🐞

• Fixed an issue for Question sets in Label management page where the question type dropdown was cropped.

• Fixed an issue where a very long display name was overflowing on the account settings page.

• Fixed an issue where model id doesn't show up in Datasaur Dinamic after done training for AWS SageMaker

• Fixed an issue where the file transformer editor was getting cut off when long error messages were displayed.

• Fixed an issue where the filter icon was not working properly in the status column of every project table.

• Fixed an issue where the size of the scroll bar for PDF was inconsistent in the Labeling Guidelines extension.

• Fixed an issue where the users were able to submit answers when the question set in Row Labeling had not yet been reloaded to the newest one.

API changes 💡

• We have deprecated name field inside QuestionInput and Question type

  • If you still have name field inside QuestionInput and Question type, either remove it or replace it with label field

  • name field inside QuestionInput and Question type will be fully removed on January 8, 2024

6.40.0

November 30, 2023

What's new 💡

• We’ve refined our project tags for improved clarity and ease of understanding.

• We've integrated Datasaur ML Assisted with LLM Lab Model Provider. This enables you to directly utilize your LLM Model, deployed from LLM Lab, in ML Assisted to assist with your labeling in Datasaur.

• We’ve improved our area selection for Tamil language in Span Labeling.

Bug fixes 🐞

• Fixed an issue where editing and replacing the label set did not update the labels already applied bounding box labeling projects.

• Fixed an issue where adding space as a label class is allowed.

• Fixed an issue in Span Labeling when applying more than 10 labels to a span will cause the rest of the labels to be black-colored.

• Fixed an issue in completed Span Labeling project where the label status changed from conflicted to accepted after Reviewer press backspace.

• Fixed an issue in Task Settings where "Enable multiple labels" option doesn't automatically apply the changes.

API changes 💡

• Marking launchTextProjectAsync mutation as deprecated. We are introducing a new createProject mutation. Removal is slated for June 30th, 2024

  • How to adjust: follow our gitbook and / or reach out to us via [e-mail]

Self-Hosted Changes 💡

• Upgrade Redis Docker image version due to security vulnerability.

6.39.0

November 23, 2023

What's new 💡

• We’ve made some improvements to the Label Set dialog design to create a more intuitive interface.

• We’ve made some improvements to the UI components to create a more intuitive interface.

• We’ve polished the Members page to a new design for a more intuitive user experience.

Bug fixes 🐞

• Fixed an issue in Labeler mode where a dialog informing the user to refresh the page didn't appear after a Reviewer changed the task settings.

• Fixed an issue in Project Creation Wizard step 3 where the question type dropdown list was being cropped when scrolling down the page.

• Fixed an issue where the request demo form continued to open when a user signed in through SSO.

6.38.0

November 16, 2023

What's new 💡

• Launched gamification onboarding to enhance user familiarity with Datasaur application upon joining.

• We've integrated OpenAI Whisper as a new Automatic Speech Recognition (ASR) system for our audio labeling project.

• We’ve improved the copywriting for the locked document message to enhance clarity.

• We’ve made some improvements to the UI components to create a more intuitive interface.

• We’ve polish the file transformer button and dialog to a new design for a more intuitive user experience.

Bug fixes 🐞

• Fixed an issue in Label Management where question is not added and removed properly after adding a new question.

• Fixed an issue where changing the selection mode setting doesn't take effect immediately.

• Fixed an issue where the error message remained visible even after a user modified the name of a duplicated question.

• Fixed an issue where the label doesn’t automatically apply to the span.

• Fixed an issue where the position of the label shifts to the right when “Enable checkboxes in label box” setting is enabled.

• Fixed an issue where the 'Project has been marked as complete' snackbar appeared differently.

• Fixed an issue where there was a mismatch in the position of blue balls on the activity page.

• Fixed an issue where there was no error message for column number validation in Excel files during Project Creation Wizard Step 1.

• Fix button spacing for “Accept All” and “Reject All” in ML Assisted after predicting labels.

API changes 💡

• In insertSentence mutation, we removed the last parameter metadata.

  • If you often call this mutation directly, or if you have an automated script that calls this mutation, you can simply omit the last parameter.

  • We will remove this parameter completely on December 18, 2023.

6.37.0

November 09, 2023

What's new 💡

• By default, registered users via SAML will be prevented to set password which is configurable by the Admin of the Workspace.

• Now, you’ll see the ‘Generate OAuth’ option in your account settings.

• We've improved the Label Management page to make the interface more intuitive.

• We’ve made some improvements on our character selection to make it more accurate for several languages.

• We’ve made some improvements to the UI components to create a more intuitive interface.

Bug fixes 🐞

• Fixed an issue in span labeling where users can’t delete a line after inserting and deleting a line.

• Fixed an issue where a project was automatically created when clicking 'Learn more about the masking data options' without launching the project.

• Fixed an issue where a user could create a document labeling project with an empty CSV file.

• Fixed an issue where changing the question set from other labelers in row and document labeling displayed an unclear error message.

• Fixed an issue where expired free trial users were still able to create a project via the file menu.

• Fixed an issue where Label All feature in the Search extension is not working properly.

• Fixed an issue where there were inconsistent project tags when creating a project.

• Fixed an issue where uploaded filenames were overflowing on Project Creation Wizard.

• Fixed an issue where user can't select or replace hierarchical dropdown answer by mouse after resizing text area box.

• Fixed an issue where the input fields in extensions are diverse

API changes 💡

• We've added new queries getProjectDocumentQuestionSet and getProjectRowQuestionSet

  • These queries return the same questions as getDocumentQuestions and getRowQuestions

  • But with an extra return value signature, which is a unique string representing the questions and their configurations.

• An optional arg questionSetSignature was added to the following mutations. This signature is what you get from getProjectDocumentQuestionSet or getProjectRowQuestionSet.

  List of Mutations:

  • updateDocumentAnswers

  • deleteDocumentAnswers

  • updateDocumentQuestions

  • updateDocumentQuestion

  • modifyDocumentQuestions

  • updateRowAnswers

  • updateMultiRowAnswers

  • insertMultiRowAnswers

  • deleteRowAnswers

  • updateRowQuestions

  • updateRowQuestion

  • modifyRowQuestions

  Unless you often send requests to these mutations directly, you may disregard these changes.

6.36.0

November 02, 2023

What's new 💡

• In our ongoing effort for clarity and consistency, we've refined our terminology. Formerly, "token" referred to text segments, ranging from one character to multiple words. Now, we'll use "span" to describe them.

  • This shift from "Token Labeling" to "Span Labeling" ensures a more intuitive experience, whether you're labeling a single word, a phrase, or individual characters. The updated term "span" will be applied platform-wide, simplifying your labeling process and enhancing the clarity of our tool's documentation.

• Label set modification is now available in bounding box labeling projects. You can upload your own label set or edit the existing the label set within the project workspace.

• New update on ML Assisted component UI.

• We reorder the sidebar items with better menu grouping.

• We've added a link to the pricing page in the account settings to enhance the user's access to information.

Bug fixes 🐞

• Fixed an issue where the capability to add label classes behaved unexpectedly after performing it in an active empty label set.

• Fixed an issue where the labeler's answer wasn't saved when labeling rows in the URL view.

• Fixed an issue where Member dropdown in Activity page only showed 10 members.

• Fixed an issue where the document status didn't consistently behave the same way between the “List of Files” extension and the “File” menu.

• Fixed an issue where clicking an item in row labeling ML Assisted answers would cause the page to automatically switch to the document labeling extension.

  • Now, when you click an item in row labeling ML Assisted answers, it will simply move the line in the editor without automatically switching the view to the document labeling extension.

• Fixed an issue where exporting Krippendorff’s Alpha resulting the calculation data of Cohen’s Kappa on the Inter-Annotator Agreement page.

• Fixed some issues in the Action: Create Project forms, the fixed issues are listed below:

  • Remove the redundant number on the form, retain the number with icon.

  • The correct error message on missing field.

  • User can select tags from dropdown field and also make a new one.

  • Project Template and External Object Storage show up when first loading the wizard.

  • Error message should have been removed after there is at least one labeler by adding a new assignee or changing the role of an existing assignee. The same goes for the project distribution.

  • Step 3 wizard sidebar is properly displayed, just like the previous steps.

6.35.0

October 26, 2023

What's new 💡

• Enhanced label modification settings to better differentiate them based on the project type.

• Enhanced question set experience by allowing user to edit the question set through the Document Labeling extension.

• Enhanced the dialogs in Members page to be more intuitive.

• Improved your experience of assigning new labelers/reviewers in Project Creation Wizard step 4 by adding a search field in the dynamic reviewer section.

• Made some improvements to the UI components to create a more intuitive interface.

Bug fixes 🐞

• Fixed a bug that allowed duplicated label set names with extra spaces.

• Fixed a bug where a tooltip for Radio button questions had a minor margin issue.

• Fixed a bug where labelers' answer status was not updating correctly in document labeling.

• Fixed a bug where the “disabled” state in some question forms was inconsistent.

• Fixed a bug where the “Mark document as completed” icon was placed following the file name.

• Fixed a bug where the “Question Set Label” copy was wrongly placed in Project Creation Wizard Step 3.

• Fixed a bug where the sorting of the Slider column was not working properly.

• Fixed an issue that caused weird scrolling in Project tags section.

• Fixed some issues in the Action: Create Project forms:

  • Remove the redundant number on the form, retain the number with icon.

  • A better and synced error messages on multiple fields.

  • User can select tags from dropdown field and also make a new one.

  • Project Template and External Object Storage options are ready to be selected when first loading the wizard.

  • Fix step 3 of the wizard sidebar UI.

• Fixed an issue where exporting Krippendorff’s Alpha resulting the calculation data of Cohen’s Kappa on the Inter-Annotator Agreement page.

API changes 💡

• We have deprecated the DASHBOARD_EXTENSION_ID inside the ExtensionID enum.

  • Dashboard Extension has long been unused in the labeling project UI and will be removed completely on November 27, 2023.

  • If you have a script to create a Datasaur project via API, you can simply omit DASHBOARD_EXTENSION_ID from your scripts.

• We have plans to enhance the Project Template feature, which will involve some adjustments to the queries related to project templates.

  • The following queries are now deprecated:

    • getProjectTemplate

    • getProjectTemplates

  • We’ve introduced new queries for the replacement:

    • getProjectTemplateV2

    • getProjectTemplatesV2

  • Comparison between the old and new queries:

    • Request arguments remain the same:

      • Both getProjectTemplate & getProjectTemplateV2 use id

      • Both getProjectTemplates & getProjectTemplatesV2 use teamId

    • Fields in old queries are not available in the new queries:

      • teamId

      • team

    • New fields in the new queries:

      • type

      • description

      • imagePreviewURL

      • videoURL

6.34.0

October 19, 2023

What's new 💡

• Introducing SAML 2.0 integration, now available for each Workspace. You can seamlessly integrate an Identity Provider into each Workspace, enabling users to sign in and register via SAML.

• Clicking Datasaur icon from Account Settings page will now return you to your last visited workspace.

• Enhanced filter functionality in both the project and archived tables for a more intuitive experience.

Bug fixes 🐞

• Fixed a bug where sorting and filtering the Time column in specific formats would display incorrect results.

• Fixed a bug where the draft saved icon would appear in newly added projects with specific question types.

• Fixed a bug where the height of the Labels’ extension would behave oddly after resizing the extension.

• Fixed a bug where the “Recently used guidelines” option could still display unsupported guideline formats.

• Fixed a bug where undoing answers in row labeling didn’t work as expected.

6.33.0

October 12, 2023

What's new 💡

• Enhanced sorting functionality in both the project and archived tables for a more intuitive experience.

• Enhanced the empty state in label set preview with a more intuitive design.

• Enhanced the box component in every page with a more intuitive design.

• Now Predict Labels will not replace existing user answers and we now facilitate Undo/Redo action for Predicted Labels

• Refined the "Mark as Complete" copy for clearer understanding.

• You can now view the IAA calculation from two algorithms: Krippendorff’s Alpha and Cohen’s Kappa.

Bug fixes 🐞

• Fix inconsistent Provider Setting in Datasaur Dinamic

• Fixed an issue when unselect answer and consensus answer in reviewer mode.

• Fixed an issue when changing the slider question type's value via the slider thumb.

• Fixed an issue when creating a duplicate label set in Project Creation Wizard.

API changes 💡

Analytics

• Deprecate getPaginatedChartData, we will remove the support for paginated data charts in the future.

• Deprecate getLabelAccuracyChart, now getCharts includes Label Accuracy chart.

• Deprecate getLabelAccuracyData, now we can use getChartData with chartId=ACCURACY_PER_LABEL to get Label Accuracy chart data.

• Deprecate exportLabelAccuracyChart, now we can use exportChart with id=ACCURACY_PER_LABEL to export Label Accuracy chart data.

• BREAKING ChartSet, now Chart type contains Array of ChartSet

  • ChartSet.BOTH is removed.

  • ChartSet.ELASTIC is introduced.

  • If a Chart included in two set, for example NEW and ELASTIC, the Chart.set property will be [NEW, ELASTIC].

Inter-Annotator Agreement (IAA)

• Deprecate calculateAgreementTables. Calculate agreement tables is a step in IAA calculation methods. It should be one with calculate IAA.

  • How to adjust: Please use calculateIAA instead.

• Deprecate calculatePairKappas. Pair kappa is a specific term to one of the IAA calculation methods. Now, Datasaur has two IAA methods.

  • How to adjust: Please use calculateIAA instead.

• Deprecate getIAALastUpdatedAt. It is IAA’s last updated time. So, it cannot be separated with IAA information.

  • How to adjust: Please use getIAAInformation instead.

• Deprecate getPairKappas. Pair kappa is a specific term to one of the IAA calculation methods. Now, Datasaur has two IAA methods.

  • How to adjust: Please use getIAAInformation instead.

• Add calculateIAA. It is added to calculate the IAA using all of the available calculation methods.

• Add getIAAInformation. It is added to improve the IAA information retrieval from multiple calculation methods.

6.32.0

October 05, 2023

What's new 💡

• Enhanced the empty state in row labeling with a more intuitive design.

• Enhanced the typography component to make it more intuitive.

• We've added tags to the onboarding project, making it easier for you to filter based on specific tags.

• We’ve enhance your experience with a new registration page design.

• We've enhanced the Project Creation Wizard by removing unnecessary vertical spaces for a clearer view.

Bug fixes 🐞

• Fixed an issue where ‘Delete Line’ tooltip in span labeling is not displaying properly.

• Fixed an issue where filtering ‘Date’ question column with the AM/PM format is not showing appropriate results.

• Fixed an issue where labeling extensions is resizable in read-only mode.

• Fixed an issue where ‘Text’ question name is missing in Project Creation Wizard.

• Fixed an issue where ‘Time’ question with default value in seconds format is not displaying appropriate

6.31.0

September 27, 2023

What's new 💡

• Experience an enhanced labeler and reviewer filter on the Project page.

• We now support sort and filter capability for Grouped Attributes question type in the row labeling projects.

• You can now manage comment email notifications in project settings.

• You can now upload PDF for the labeling guidelines.

Bug fixes 🐞

• Fixed an issue where filtering time question doesn't behave properly.

• Fixed an issue where importing a file transformer using the library did not immediately update to the new one.

• Fixed an issue where the comment tab kept closing when opening another comment in row labeling project.

• Fixed an issue where undoing rejected labels in the reviewer mode doesn't behave properly.

6.30.0

September 21, 2023

What's new 💡

• We've added a button to improve IAA calculation inside Data Programming. It'll enable you to quickly recalculate the IAA between Labeling Functions if needed.

• We've improved the tooltip feature in the reviewer mode. Now, when you hover over long answers, a vertical scrollbar will appear, making it easier to review content.

• We've added video tutorials and comprehensive documentation to the Help menu. This will assist you in navigating and using our platform effectively.

• Labeler Mode now will always show first when you open a project where you are both the labeler and reviewer of the project.

• We've released Datasaur Dinamic using AWS Sagemaker for row labeling project.

• We’ve improved the delete and archive dialog to provide clearer and more helpful information.

• We’ve enhanced your experience with member assignment by auto-assigning to the project when there is only one member in the workspace.

• You will now receive email notifications for any active comment threads in your assigned project.

• You can now seamlessly integrate Azure Blob Storage with External Object Storage.

Bug fixes 🐞

• Fixed an issue causing empty rows to appear when sorting the Textfield question column in ascending order.

• Fixed an issue where clicking on a URL answer would open a new tab instead of selecting it as the answer.

• Fixed an issue where sorting the Dropdown question column was behaving strangely and causing confusion.

• Fixed an issue where the labeling status on the Member tab was not automatically updated when a reviewer changed the project status.

• Fixed an issue where uploading a question set with duplicate question labels was still allowed.

• Fixed an issue when users were able to add comments in document labeling projects while using .csv and .tsv file.

6.29.0

September 14, 2023

What's new 💡

• Datasaur Predictive Labeling improvement on the UX : Showing predicted answers for Row Labeling Project.

• Datasaur University will now be called Video Tutorials for better clarity.

• Empty text transcriptions uploaded for audio labeling will now have a preserved new line.

• Enhanced ease of use and your overall experience by adding three new options to the Help menu: video tutorials, release notes, and contact support.

• Redesigned the Document Labeling extension to make it more intuitive.

• Now, we have set a maximum limit 100 assignees for each project.

• We have enhanced the sample projects list by organizing it chronologically.

Bug fixes 🐞

• Fixed an issue where changing the pagination number couldn’t be accomplished by pressing Backspace.

6.28.0

September 08, 2023

What's new 💡

• Now all project templates only show the supported format for each template.

• You can now customize colors when setting up the Slider question type

Bug fixes 🐞

• Fixed an issue that caused an ambiguous error when updating the file transformer name that exceeded the maximum character count.

• Fixed an issue where switching question sets containing Text to Slider caused an invalid draft answer.

• Fixed an issue where trial form keeps reappearing after filling the form.

6.27.0

August 31, 2023

What's new 💡

• Redesign certain snackbars to be more intuitive.

Bug fixes 🐞

• Fixed an issue that caused an unclear error message to be displayed when uploading radio buttons, dropdowns, and hierarchical dropdowns with no options.

• Fixed an issue that caused comment indicators disappeared after adding a new line in span labeling projects.

• Fixed an issue that validates empty regular expression in Data Masking Settings.

• Fixed an issue where email notification was sent when project creation was failed.

• Fixed an issue where ripples caused unintended behavior when reordering question types.

• Fixed an issue where the row filter and page navigation in the URL viewer project didn’t behave appropriately.

• Fixed an issue with the Labelers' icon showing a blue circle around it when the project is marked as complete.

6.26.0

August 24, 2023

What's new 💡

• A new look in our Inter-Annotator Agreement (IAA) Page, now showing a fresh and revitalized look that enhances user experience and aesthetics.

• Enhance the visual appearance of the file transformer editor.

• Revamped the Question Set dialog on the Label Management page to provide clearer and more helpful information.

Bug fixes 🐞

• Fixed an issue that caused incorrect results when sorting columns in Date or Month questions with more than 10 values.

• Fixed an issue where a Dropdown question with no options could still be used.

• Fixed an issue where the “Save to Library” option was displaying unintended suggestions for Label Set names.

• Fixed an issue where holding the CTRL key and clicking the same token resulted in an incorrect selected range.

• Fixed an issue where the tooltip in the navigation bar on row labeling was causing a blinking effect.

• Fixed an issue where column sorting and the displayed row page were showing incorrect results.

• Fixed an issue where a comment button was appearing in document labeling projects.

• Fixed an issue where the comment button wasn't showing up in row labeling projects.

• Fixed an issue where the selection areas for checkboxes in the question set were not aligned properly.

6.25.0

August 16, 2023

What's new 💡

• The Extension panel will now automatically collapse after disabling all extensions.

• We add the latest time information when the IAA data is updated. This information perfectly matches the time when the last labeler marked the document as complete.

• We have enhanced the tooltip for a long project name.

• We simplify your labeling process through automation, Datasaur Predictive Labeling, by providing us with a mere five data samples.

Bug fixes 🐞

• Fixed an issue that allowed users to upload answer set with duplicate answer label for Dropdown, Hierarchical dropdown, and Radio question types.

• Fixed an issue that displayed incorrect questions when saving the question set to the library.

• Fixed an issue where holding Shift after applying labels would result in selecting more text than intended.

• Fixed an issue where page navigation doesn’t work properly when the row filter is activated.

• Fixed an issue where the context menu wouldn’t appear upon clicking “Edit” in the table viewer header.

• Fixed an issue where users were able to switch between labeler and reviewer modes even after changing the project status to "Mark Project as Complete”.

• Fixed an issue where users were unable to switch to labeler mode after changing the project status from "Mark Project as Complete" to "In Review”.

6.24.0

August 10, 2023

What's new 💡

• Add a Cumulative Time Spent chart, which shows the total time spent by both labelers and reviewers each day working on projects, calculated when opening the Datasaur project page as the current active tab of the browser.

• Experience a rejuvenated look on the Analytics Overview page, complete with the addition of tabs to easily navigate between the Overview data and the Inter-Annotator Agreement (now thoughtfully relocated from the sidebar).

• Introduce the Remaining Files chart, categorized by project status to empower you to easily assess pending tasks across different projects.

• Now, we will no longer support Visualization (pdf) as an export format.

• Renaming JSON Deprecated to JSON Simplified for better clarity.

• Unify previously segmented table data from the Overview page into the Projects page, providing a holistic snapshot of project progress. You can now delve into individual Labeler progress by simply hovering over their icon within the Project table.

• We have enhanced the page navigation capability in row labeling projects, making it easier to move between pages.

• We have enhanced the row labeling capability by adding the option to filter rows based on unreviewed and reviewed answers.

Bug fixes 🐞

• Fixed a bug that was causing issues with the discard draft capability during the second attempt.

• Fixed a bug that was preventing the discard capability from working properly for the Radio question type.

• Fixed a bug in the Project Creation Wizard where the Dropdown options in the question set preview were not being displayed.

• Fixed a bug that allowed users to edit and save question sets with duplicated question set names.

• Fixed an issue where labels were being cropped at the top of the page/image.

6.23.0

August 3, 2023

What's new 💡

• Applying multiple labels to the same spans is now easier and more intuitive.

• We have improved the Hierarchical Dropdown question type by adding the capability to select only the leaf options.

• You can now set default Zoom preferences for media within projects that require PDF files.

Bug fixes 🐞

• Fixed an issue where adding new label classes in the label box would cause the arrows to disappear.

• Fixed an issue where unselecting rows on the limited rows per page interface would result in the rows not being displayed at all.

• Fixed an issue where the text is cut when creating projects with multiple answer setting unselected.

API breaking changes ⚠️

• Changing the documents attribute on Cabinet type to a scalar. The reason for this is to improve performance and load times, e.g. getProject query with lots of documents.

  • How to adjust: stating only the documents attribute when fetching a cabinet is enough now, no need to specify the detail attributes anymore.

6.22.0

Jul 27, 2023

What's new 💡

• Add a new Label Set segment on Custom Report Builder.

• Reviewers will now receive alerts when marking the project as complete.

• The Label Set extensions have been enhanced with a sleeker design.

• We now provide a linear progress bar below the toolbar to indicate the progress of recalculation.

Bug fixes 🐞

• Fixed an issue where filtering columns by Date question in row labeling projects resulted in incorrect output.

• Fixed an issue that caused the comment button to not work properly in row labeling projects.

• Fixed an issue that caused duplicate label IDs when creating prelabeled span labeling projects.

• Fix and improve Team Member Analytics table calculation for both Labeler and Reviewer.

6.21.0

Jul 20, 2023

What's new 💡

• Reviewers now can receive email notifications when certain numbers of labelers have finished their labeling projects.

• We now support a radio button as a new question type in both row labeling and document labeling projects.

• We have differentiated the UI style for ML Assisted and human-generated labels, improving the user experience in row labeling project.

Bug fixes 🐞

• Fixed an issue where the answer draft was not discarded properly for dropdown, hierarchical dropdown, date, and time questions.

• Fixed an issue where the screen would go blank when labelers submitted a checkbox answer in a row labeling project.

• Fixed an issue where the screen would go blank when playing an audio file using a specific file transformer.

• Fixed an issue where project template icons were not updated after refreshing the page.

6.20.0

Jul 13, 2023

Bug fixes 🐞

• Fixed an issue where adding a new option to the dropdown question caused the answer draft of the previous rows to disappear.

• Fixed an issue where an error message was still displayed after removing a broken file.

• Fixed an issue where there was no clear divider between labelers’ names and their answers on the tooltip in the Reviewer mode.

• Fixed an issue that caused an export failure when enabling PII and exporting to Visualization format.

6.19.0

Jul 6, 2023

What's new 💡

• Error message for false document assignment has been improved for better clarity.

• Improve the Team Member Analytics charts, making it aligned with the Overview page.

• Manage question logic dialog in Project Creation Wizard Step 3 has been redesigned to enhance its intuitiveness.

• We have made labeling even better by reducing the hassle of refreshing the page when new label classes are added, making your experience smoother and more convenient.

• We now support mixed labeling projects of bounding box and document using prelabeled documents.

Bug fixes 🐞

• Fixed an issue where some documents were not assigned in a multi-document project.

• Fixed an issue where a portion of copywriting showed twice in an email sent after exporting projects.

• Fixed an issue that caused an error when creating row labeling projects with Chinese characters within the URL.

• Fixed an issue that caused wrong project type when creating a project with a Datasaur Schema format containing one column.

• Fixed an issue that caused the user's prompt to not get saved when refreshing the page.

6.18.0

Jun 27, 2023

What's new 💡

• Adding Inter-Annotator Agreement in Data Programming to measure how well Label Functions can make the same annotation decision

• Now, you can easily mark a document or project as complete by simply use the shortcut or find the "Mark as Complete" button under the File menu.

• The Slider question type has been enhanced with the ability to show/hide the value and provide more question logic options.

• We now support sort and filter capability for Hierarchical Dropdown question type in the row labeling projects.

• We now provide an email to our documentation and exciting Youtube tutorial upon verifying an account.

• You can now see pre-labeled information in label’s tooltip.

Bug fixes 🐞

• Fixed an issue for clickable area in Project Creation Wizard to open documentation about task types

• Fixed an issue where removing questions that have logic caused an error to occur.

• Fixed an issue where users were able to submit empty answers for required questions with multiple answers enabled.

• Fixed an issue where unchecking labelers’ answers in the form of URL doesn’t clear the field.

• Fixed an issue where the Dropdown question option overflowed the Document Labeling extension.

6.17.0

Jun 22, 2023

What's new 💡

• Add restriction for Hugging Face Provider in Datasaur Model Training to only support dropdown-type questions.

• Add new provider Amazon Comprehend for ML Assisted Extension! Now user can use their Amazon Comprehend model endpoints natively in our ML Assisted extension.

• Enhanced Analytics Overview with new charts for improved insights, while still providing the option to revert to the previous version.

• Revamp Project Analytics, just like Overview, making sure the data is aligned.

Bug fixes 🐞

• Fixed an issue where creating questions with logic could not be done in the Label Management page.

• Fixed an issue where the draft answers not cleared after submitting answers in row labeling project.

• Fixed an issue where removing questions that have logic would cause an error.

• Fixed an issue where resizing the Document Labeling extension would reload the extension and discard the draft answers in row labeling project.

• Fixed an issue where zooming in on the last page of a PDF would automatically jump to a certain page.

• Fixed an issue where an out-of-date error appeared when exporting personal projects.

6.16.0

Jun 15, 2023

What's new 💡

• Label Management page has been polished with a new empty state design.

• The error message for Datasaur Model Training has been refined and improved.

• The interface and interaction in the URL viewer have been enhanced with the capability to copy links.

• Unfinished answers in row and document labeling now will be saved automatically as a draft.

• Zoom-in experience for documents or images has been polished.

Bug fixes 🐞

• Fixed an issue where label classes in the Labels extension looked clickable.

• Fixed an issue where the comment button not scrolling to the selected row.

6.15.0

Jun 8, 2023

What's new 💡

• Added support for Multi-label answers in span labeling projects. Users can now create labeling functions with multiple label answers (dropdown selection).

• Added the two-factor authentication (2FA) option to ensure the security of your account.

• Exciting interface update for label and question set management menu.

• Now, you will see the labeler or reviewer mode based on your last state in the project.

• The submission of answers in row labeling project has been enhanced by lowering the color opacity to indicate that the answers are in the process of being submitted.

• We now support sort and filter capability for Dropdown question type in the row labeling projects.

Bug fixes 🐞

• Fixed an issue that caused a rendering error for questions in the Document Labeling extension in the personal workspace.

• Fixed an issue where the project name did not use the new UI design.

• Fixed an issue that caused Label Count in the Playground page not updated.

• Fixed an issue where there is an additional row added in row labeling project.

• Fixed an issue where CSV file are accepted for a project that already has TXT files.

• Fixed an issue where warning appears for consensus even though has met the consensus number.

6.14.0

May 31, 2023

What's new 💡

• Conveniently open the package page in a new tab for free trial.

• Easily access support and documentation from error message pop-up when project creation fails.

• Exciting new icons added to enhance your project template.

• File upload in Document Labeling extension has been redesigned to be more informative.

• Filtering column by Date question type experience has been enhanced with a more intuitive error handling.

• Label Set extension has been redesigned to enhance its intuitiveness.

• Release Datasaur Model Training BETA version. This will help user to connect with Hugging Face Auto Train.

• Transform the value of the Time Spent metrics in Custom Report Builder to Minutes.

Bug fixes 🐞

• Fixed an issue that caused the answer field in the Document Labeling extension was not cleared after labelers clicked the Submit button.

• Fixed an issue where Textarea and URL question types had different widths in the Project Creation Wizard Step 3.

• Fixed an issue where infinite loading was displayed at the bottom of the table within a document labeling project containing CSV files.

• Fixed an issue where long label set names were overflowing in the label set list within the Label Sets extension.

• Fixed an issue where name does not immediately change after filling Full Name pop-up.

6.13.0

May 25, 2023

What's new 💡

• PDF viewer has been enhanced with a page counter that updates accordingly.

• We now support hierarchies of labels up to 10 levels in span labeling projects.

• We now support sort and filter capability for Slider question type in the row labeling projects.

• Now, you will be notified for the free trial period through our apps and email.

• Now, you can easily access our documentation for the accepted format directly from the Project Creation Wizard page.

• The labeling experience in the Bounding Box labeling project has been improved with a wider label box space for better text visibility.

Bug fixes 🐞

• Fixed an issue that caused flickering tooltip for long labels in the label options menu.

• Fixed an issue where the labeler mode screen does not load properly.

6.12.0

May 17, 2023

What's new 💡

• Empty state of Document Labeling extension has been redesigned to be more informative.

• Review extension has been redesigned to be more intuitive with better information hierarchy.

• You can now have smoother experience when applying labels.

• You can now see your package plan inside the Account settings menu.

• You can now directly use the export result for model training on Hugging Face.

• You can now directly use the export result for model training on Amazon Comprehend.

Bug fixes 🐞

• Fixed an issue where column menu and right-click menu were stacked.

• Fixed an issue to reject XLSX file as labeling guidelines in old projects.

• Fixed an issue that caused Admin chip to be cut off when the name was too long.

• Fixed an issue where a CSV file contains empty question names were allowed to upload.

• Fixed an issue where jumping to the first row occurred when attempting to add the first comment in a row labeling project.

6.11.0

May 11, 2023

What's new 💡

• Adjusted the Open AI temperature range to ensure focused and accurate answers for ML-Assisted usage.

• Improvement to our ML-Assisted with Open AI Provider feature. Now, if an error occurs with a specific document row, you don't have to worry about the entire process being canceled. We've enhanced our system to display the successfully applied label, ensuring a smoother and more efficient labeling experience.

• Some sections have been redesigned to be more intuitive and easy to use:

  • Comment popover on token editor and table viewer.

  • Search extension.

• We now support sort and filter capability for Time question type in the row labeling projects.

• You can now directly use the export result for model training on GCP Vertex AI.

• You can now directly use the export result for model training on Azure AutoML.

• You can now select an option to delete the backup file after it's successfully uploaded to an object storage.

• You now have the ability to mark all documents as complete when marking a project as complete.

Bug fixes 🐞

• Fixed an issue that caused wrong conflict count in Review extension.

• Fixed an issue where several keyboard shortcuts on Mac didn’t work as expected.

• Fixed an issue that caused the appearance of old answers from labelers after replacing question set.

• Fixed an issue where files can be split more than the maximum value in Split Files feature.

• Fixed an issue where File Transformers not saved correctly.

6.10.0

May 4, 2023

What's new 💡

• We now support sort and filter capability for Checkbox and Date question type in the row labeling projects.

• We now offer seamless native integration with Google Vertex AI, allowing you to predict labels using your existing model with ease.

• You can now see tutorial videos in Datasaur University under the User Account menu dropdown.

• You can now retrieve more than 2,000 rows of data when generating the report with the Custom Report Builder.

Bug fixes 🐞

• Fixed an issue where the document conflict wasn’t updated after reviewing a question.

• Fixed an issue that caused infinite loading when opening an empty document labeling project.

• Fixed an issue where you can not apply label in Bounding Box Labeling project when using HTML color names for the color in label set.

• Resolved the problem that prevented the correct display of excessively lengthy questions within question sets.

• Resolved an issue where data programming was not refreshing after deleting labeling functions.

6.9.0

April 13, 2023

What's new 💡

• Datasaur Free Trial

  • Are you interested to explore full features of Datasaur? Now we are available for a limited 14 days of free trial.

  • Are you a free user of Datasaur? You can now create one team workspace to explore our features more.

• Some sections have been redesigned to be more intuitive and easy to use:

  • Sort and filter menu of Text Field and Text Area question type.

  • Members tab of Review extension.

• We now support sort and filter capability for URL question type in the row labeling projects.

• We now support External Object Storage with buckets from AWS regions that are disabled by default.

• We now offer support for native integration with Azure AutoML, allowing you to easily predict labels using your existing models in Azure.

• You can now use file transformer for row and span labeling projects by using API.

• You can now reorder questions from Question Set Management menu.

Bug fixes 🐞

• Fixed an issue where projects were hidden after saving Manage Tags menu.

• Fixed an issue where “Add comment” button didn’t work on currently selected tokens and went to the first token.

• Fixed an issue where triple dot menu from Labeling Guidelines extension didn’t hide after the file successfully uploaded.

• Improved error messaging for OpenAI API usage to provide clearer and more informative feedback to users.

6.8.0

April 3, 2023

What's new 💡

• You can turn on or off the labeling behaviour of moving to the next span after applying labels from Project Settings dialog.

• You can create a prelabeled Bounding Box projects by using .answer.json file format.

Bug fixes 🐞

• Fixed an issue on missing Action Runs button after changing the page.

• Fixed an issue that caused error when deleting questions from an uploaded question set with logic applied.

• Improved error message when a labeling function of data programming returns an invalid value.

6.7.0

March 30, 2023

What's new 💡

• Add the Project Template creation limit to 12 and improve the scrolling behavior so that you can see all the Project Templates.

• Datasaur text style has been updated to provide better readability and fresher look.

• You can upload JSON file in the Label Set extension.

Bug fixes 🐞

• Fixed an issue where the tooltip of right-click menu was shown behind the menu.

• Fixed an issue that caused the wrong document to show when switching between Labeler Mode and Reviewer Mode.

• Fixed an issue that caused Dropdown questions looked squeezed when saving a question set to the library.

• Fixed an issue on right-click issue in the TEST project.

• Fixed an issue where clicking outside label box area didn’t save the bounding box label and caption.

• Fixed an issue that caused weird behavior in the Dynamic Question Set for answers containing characters.

• The Analytics data is updating normally, which is in every 3 hours.

• The Action Run Detail now shows the correct labelers that are assigned in the project.

6.6.0

March 23, 2023

What's new 💡

• Comment popover has been redesigned to be more intuitive and easy to use.

• Generate OAuth dialog has been redesigned to be more intuitive and easy to use.

• Switch modes menu from profile dropdown has larger clickable area now to reduce the potential of misclicking the menu.

• You can now automate project creation using files directly from your own object storage using the Action: Create Project. It would be very convenient if you are creating multiple projects frequently, especially with the same configuration.

Bug fixes 🐞

• Fixed an issue that caused an inactive extension to keep showing in Full Height mode.

• Fixed an issue that caused the menu not show when right-clicking on a cell in the row labeling projects.

• Fixed an issue that caused blinking labels in the span labeling projects.

• Fixed an issue where the file progress didn’t change after drawing bounding box in a multiple files Bounding Box labeling project.

• Fixed an issue that caused an empty answer from Grouped Attributes question to show after submitting answers.

• Fixed an issue that caused the additional column to show when reimporting pre-labeled row labeling project.

• Fixed an issue that caused URL view setting was not saved when using project template

• Fixed an issue where “Clear All” action didn’t work in the project dashboard..

• Folders that are successfully processed by Action (created projects) will be removed from the object storage. It’s removed since the actual files already being moved to the target path. This process will avoid any confusion as to which folders that have been processed successfully before.

6.5.0

March 16, 2023

What's new 💡

• Add a new export format for GCP Vertex AI, specifically for Row Labeling project. You can now directly use the export result for model training on GCP Vertex AI.

• Data programming integration with OpenAI: we leverage data programming by enabling OpenAI prompt to be used as a labeling function.

• Labeling Guidelines extension has been redesigned to be more intuitive and easy to use.

• ML-Assisted labeling with OpenAI integration: now you can create prompt and get the predicted result for span and row labeling projects.

Bug fixes 🐞

• Fixed an issue where the labels were truncated due to long label set name.

• Fixed an issue where Project Creation Wizard JSON script not updated when deleting a document.

• Fixed an issue where no error message showed after failing to load CONLL file.

• Improved copywriting in the Review extension when there was no labelers assigned.

• Remove the instruction from Data Programming dialog for better visibility of its code editor.

6.4.0

2 March 2023

What's new 💡

• Add Row Labeling export format for Azure AutoML.

• Improve copywriting for Auto Caption in Project Creation Wizard and extension.

• Modify the Top Part of Review extension.

• Remove create your own Label Set extension upload state and improve the upload progress state.

Bug fixes 🐞

• Data-Programming's prediction returns nothing for Row Labeling project that only has one line.

• Empty state for completed files only filter in List of Files extension.

• Guideline Extension could still be modified after a project is completed.

• Hide/show a question column in Row Labeling project should display the latest labeling state.

• Label for Bounding Box leaks through the viewer area.

• Reviewer Extension doesn't show empty labeler state.

• Unable to create Document Labeling with Markdown file.

6.3.0

23 Feb 2023

What's new 💡

• Add validation for file naming while exporting a project.

• Disable Project Setting and Save as Template menu for Bounding Box project.

• Improve button and copywriting in Delete label set dialog.

• Improve project creation speed by delegating some process to queue.

• Support file upload with Bounding Box Label Set schema.

Bug fixes 🐞

• Cannot delete an answer.

• Cannot draw new Bounding Box labels after highlighting the caption field and the cursor is directed outside the Label Box.

• Changing the View menu to Unlabeled row unchecks rows except the last one selected.

• Disable auto-caption setting when the project is completed.

• Filter Unlabeled only does not work properly.

6.2.0

16 Feb 2023

What's new 💡

• Add new GraphQL query to get signed URLs for uploading data.

• Allow extension to be shown in full height and toggle to switch between height.

• Change scrollbar styling globally.

• Disable edit text in Reviewer Mode.

• Differentiate the state (with color) if the Labeling Function is empty on Data Programming.

• Export project result directly to an External Object Storage.

• Misleading error message when there are unintended files in the document folder.

• Update Document Labeling viewer.

Bug fixes 🐞

• Overflow Bounding Box label tooltip.

• Slow network causing Bounding Box not rendering properly.

• Some Bounding Box labels are not showing in the Reviewer Mode.

6.1.0

9 Feb 2023

What's new 💡

• Adjust project creation to support Bounding Box phase 2.

• Enable the Select All Rows checkbox when filtering columns by keyword or filtering in the View menu is enabled.

• Implement Bounding Box Auto Label.

• Make Question and Answer ID index start from 0.

Bug fixes 🐞

• Disable resize/move OCR label after the project is completed.

• Invalid activationConditionalLogic if the Question Set contains bindToColumn.

• Stuck on resize after selecting label in Bounding Box Labeling.

• Unhandled error when uploading TSV files with a line that has different column count.

6.0.0

2 Feb 2023

What's new 💡

• Create Account Setting page to update name, change/set password, and change profile picture.

• Enable Action for a specific team in Beta mode.

• Implement get prediction for Span Labeling Data Programming.

• Improve error message when project creation failed due to different columns.

• Loading state for Bounding Box Labeling.

• Update export file dialog.

Bug fixes 🐞

• Alt+K shortcut in Audio Labeling doesn't work as expected.

• Dropdown questions that allow multiple answer are not rendered correctly in Question Set Management page.

• Invalid argument error occurred when reimporting Datasaur Schema from Row Labeling project.

• White space value on Label Set Item makes it broken.