Data Programming

Label data with rules and heuristics.

Introduction

The Data programming extension helps automate data labeling for large datasets using rules and heuristics. It reduces manual effort and improves labeling efficiency. You can define labeling functions, create rules, and apply patterns to label data more systematically.

Key features

• Labeling functions: Use rule-based functions to generate labels. Individual functions may be imperfect, but combined they improve overall prediction quality. Labeling functions can be written in Python using the provided template.

• Labeling function analysis: View metrics such as coverage, overlaps, and conflicts. Use these insights to improve performance and train the label model. Supports both span labeling and row labeling project.

• Inter-annotator agreement for labeling functions: Measure agreement between labeling functions and reviewed answers to evaluate performance.

Supported libraries

If you need additional libraries, please reach out to us by contacting [e-mail].

Name	Version
pandas	1.4.4 and later
textblob	0.17.1 and later
nltk	3.7 and later
spacy	3.4.1 and later
scipy	1.9.1
numpy	1.23.3
transformers	4.28.1
requests	2.28.1 and later
datasets	2.7.0
openai	0.27.0
stanza	1.5.0 and later
spacy-fastlang	1.0.1 and later
lxml	4.9.2

Enable Data programming

Quick guide

1. Open Manage extensions dialog and enable the Data programming extension.

   
2. Select the question (for row labeling projects) or label set (for span labeling projects) to apply labeling functions.

3. Enable Multi-label template to use a template that supports predicting multiple labels. By default, labeling functions predict a single label. Find the sample templates here.

   
4. Create labeling functions by clicking the Labeling functions button. Use the provided Python template based on your label set.

   

Edit only the section marked Start editing here. Do not modify the surrounding code.

1. Close the Manage functions dialog and click Predict labels.

Labeling function templates

583B

Default Labeling Function for Row (1).txt

Open

653B

Default Labeling Function for Span (1).txt

Open

775B

Multiple Labeling Function for Row.txt

Open

978B

Multiple Labeling Function for Span.txt

Open

Manage labeling functions

• To create a new labeling function, click Add. A template will be generated in the code editor.

  
• To rename a labeling function, hover over the name and click the edit icon. Type the new name, then click the check icon to save.

  
• To delete labeling functions, select one or more labeling functions using the checkboxes, then click Delete. Confirm the action in the dialog.

  
• To enable or disable labeling functions, use the toggle next to each labeling function to include or exclude it from prediction.

  

Build labeling functions

Default label

By default, a labeling function assigns 1 label, defined in the following line:

#Assign target label based on LABELS dictionary
@apply_label(label=LABELS['labelA'])

Input text

By default, a labeling function processes all columns in a row:

## please check <link section 'text' in gitbook> for more info
text = list(sample.values())[0]

To use a specific single column:

text = sample[<COLUMN_NAME>]

To use specific multiple columns:

text = ' '.join([sample[<COLUMN_NAME_A>], sample[<COLUMN_NAME_B>]])

Output format

• Row labeling: Returns a boolean value.

• Span labeling: Returns a match_list, a list of matched token index ranges in the format [start_index, end_index] .

  In this example:

  >>> text = "Russian and American Alien Spaceship Claims Raise Eyebrows"
  >>> target_token = ["Russian", "American"]
  >>> match_list = [[0,7],[12,20]]

  match_list is a list of target_token positions regarding to text .

Regex

If you use regex in your logic, you can find match_list with regex.finditer :

	# *TARGET could be **keyword** or **regex pattern.*** 

>>> match_list = [re.finditer(target, text) for target in TARGET_KEYWORDS]

or

>>> date = re.compile(r"(19|20)\\d\\d[- /.](0[1-9]|[012])[- /.](0[1-9]|[12][0-9]|3[01])")
>>> PATTERNS = [date]
>>> match_list = [re.finditer(pattern, text) for pattern in PATTERNS]

Labeling function remover

• Row labeling

  def labeling_function(sample):
      return False

• Span labeling

  def labeling_function(sample):
  		
  match_list = []
  return match_list