Get Started

If there is not already a transformation block that meets your needs, one option is to write your own. Edlink allows you to write custom transformations in JavaScript/Typescript. These transformations can be used in the pipeline just like the standard ones.

Transformations are applied in order and the output of each transformation is passed as the input to the next transformation. This means that the output of your custom transformation could affect the result of, for example, licensing rules.

Creating the block

You can create custom transformations in the Data Pipeline previewer. To do so click the Add Block button in the sidebar. Then select Create A Custom Script from the list of options. This will create a new custom transformation, add it to the pipeline, and open the editor.

Generating code with AI

The Custom Script editor includes a Generate tab that allows you to create transformation scripts using AI prompts. This is useful if you are not comfortable writing JavaScript/TypeScript or want a quick starting point.

1. Inside the Custom Script editor, click on the Generate tab.
2. In the text area, describe the transformation you want to perform. Be as specific as possible.
   • Example: "Filter out all courses that do not have any enrollments."
   • Example: "Capitalize the last name of every person."
3. Click the Generate button.
4. The AI will generate the code for you. Review the code to ensure it meets your requirements.
5. The generated code will be populated in the editor. You can switch back to the Code tab to view, test, or modify the script manually.

Writing the function manually

When writing a custom transformation you can perform any operation you want on the data. The only requirement is that you return the Dataset object that you are passed. In addition to standard JS operations, you can also use the lodash library. The lodash library is already included and can be accessed via the _ namespace.

Function signature

The function signature for a custom transformation is as follows:

import { Dataset } from 'index';

function transform($ctx: Context, $config: Record<string, any>, $input: Dataset): Dataset {
    // Write your transformation function here.

    return $input;
}

When you open the transformation editor, you will see a function that looks like this. This is the simplest possible transformation. It takes the input dataset and returns it unchanged.

It is vital you use exactly this function signature. If you change the name of the function, the parameters, or even remove the types, the transformation will not work.

The $config parameter is not used currently.

The $ctx parameter contains information about the source, integration, and other metadata.

The $input parameter is the dataset that you will be transforming. It contains Maps of all the entities in your source/integration. It is self-referential and will manage references/dependencies for you. It is recommended that you modify this dataset in place and then return it for the best results.

The following are some example transformations that you can use as a starting point for your own.

Change everyone's name to "John Doe"

This transformation loops over all the people in the dataset and sets their first_name to John and their last_name to Doe and their display_name to John Doe.

import { Dataset, Person } from 'index';

function transform($ctx: Context, $config: Record<string, any>, $input: Dataset): Dataset {
    for (const person of $input.people.values()) {
        person.first_name = 'John';
        person.last_name = 'Doe';
        person.display_name = 'John Doe';
    }

    return $input;
}

Set a person's role based on their display name

This transformation loops over all the people in the dataset and sets their role to Teacher if their display_name starts with Mr., Mrs., or Ms..

import { Dataset, Person, Role, GenderIdentity } from 'index';

function transform($ctx: Context, $config: Record<string, any>, $input: Dataset): Dataset {
    for (const person of $input.people.values()) {
        if (person.display_name.startsWith('Mr.')) {
            person.roles = [Role.Teacher];
            person.demographics.gender = GenderIdentity.Male;
        } else if (person.display_name.startsWith('Mrs.') || person.display_name.startsWith('Ms.')) {
            person.roles = [Role.Teacher];
            person.demographics.gender = GenderIdentity.Female;
        }
    }

    return $input;
}

Set a person's role based on their email address

This transformation assigns roles by checking if the user's email address matches a specific pattern using Regex. In this example, any email containing a number or ending in @stu.school.edu is treated as a Student, while all others are treated as a Teacher.

import { Dataset, Person, Role } from 'index';

function transform($ctx: Context, $config: Record<string, any>, $input: Dataset): Dataset {
    // Regex for emails that contain digits or come from a student subdomain
    const studentPattern = /\d+|@stu\.school\.edu$/;

    for (const person of $input.people.values()) {
        if (!person.email) continue;

        if (studentPattern.test(person.email)) {
            person.roles = [Role.Student];
        } else {
            person.roles = [Role.Teacher];
        }
    }

    return $input;
}

Things to be aware of

• first_name and last_name are different than display_name, and they will not change together.
• Deleting entities with dependents (via the entity delete() function) will delete the dependents as well. For example, deleting a person will delete all their enrollments.
• Troubleshooting missing data: If expected data (such as People or Classes) is missing from your integration sharing, check your custom transformations. Misconfigured scripts that inadvertently filter out entities are a common cause of missing data in the final dataset.