Sandbox Environment

Importing

If there are any models that you need to use in your transformation, you can import them from the index module. For example, if you need to use the Person model, you can modify the import to look like this:

import { Dataset, Person } from 'index';

And then use the Person model in your transformation:

function transform($ctx: Context, $config: Record<string, any>, $input: Dataset): Dataset {
    const district_id = $input.districts.values().next().value.id;
    const person = new Person();

    person.first_name = 'John';
    person.last_name = 'Doe';
    person.display_name = 'John Doe';
    person.school_ids = ['00000000-0000-0000-0000-000000000000'];
    person.district_id = district_id;

    $input.people.create('external_id', person);

    return $input;
}

Or to use an Edlink enum:

import { Dataset, Role } from 'index';

function transform($ctx: Context, $config: Record<string, any>, $input: Dataset): Dataset {
    console.log(Role.Teacher);

    return $input;
}

The editor can also automatically add imports for you. If this doesn't happen by default try using the CTL + . or Command + . shortcut on the type's name. By importing the models, you can make use of type checking and autocomplete in the editor. However, they are not critical to the transformation compiling and running.

The index module contains all of the models that are available in the Data Pipeline. Because of this you can use it to understand what data is available to you as you are typing. It works just like autocomplete in your code editor. This is true even if you dont explicitly import the types. For example if you loop over the people in a dataset, you can see what properties are available on the Person model by typing person. and scrolling through the autocomplete.

Do not modify any properties prefixed with an underscore (_). These are used internally by Edlink and modifying them could cause unexpected behavior.

Sandbox

Custom transformations run in a sanboxed environment. This means that you cannot import base node modules like fs or http nor can you import any external modules. You can only import modules that are included in the index module. Currently this is comprised just of Edlink's types. The Lodash library is injected into the sandbox and can be accessed via the _ namespace without an import statement.

Additionally the sandbox is limited to 8GB of memory and 120 seconds of execution time. It runs on an isolated thread where the only input is a string (the serialized dataset) and the only output is a string (a serialized dataset). There is no network access and no access to the file system.

These limitations are in place to ensure that custom transformations are safe and only run on the Data that they are supposed to. Security is a top priority for Edlink