Event Deltas API

Using Edlink's Events API, you can observe incremental changes in provider data over time. Using this data, you can make updates to your own database. We typically call this procedure a "hot sync". This is in contrast to a "full sync" which would constitute fetching all of the data from the provider.

Event Generation Behavior

It is important to understand that events in Edlink represent the difference (delta) between two materializations of the data. A materialization occurs when data is synced from the source system and processed into the Edlink data model for your specific integration.

Initial Connections

When an integration is first connected and the initial materialization completes, Edlink generates created events for the entire dataset (every person, class, enrollment, etc.).

Paused Integrations

If an integration is set to Paused status:

1. No new materializations occur.
2. Therefore, no events are generated while the integration remains paused.

When the integration is Resumed (set back to Active):

1. A new materialization is triggered immediately.
2. Edlink calculates the difference between this new state and the last successful materialization before the pause.
3. Events are generated for all data changes that occurred during the pause period.

Because these events are generated as new events at the moment the integration resumes, they receive a fresh creation timestamp. This means that even if an integration was paused for longer than 30 days, you will still receive events for all changes that happened during that time when you resume.

You should note that events go back in time only up to 30 days, in accordance with our data retention policy. Events older than 30 days are subject to deletion, and cannot be recovered. For this reason (among others) it's important that you can also support a "full sync" with Edlink.

You can read more about how to perform a "full sync" here.

Initial Connection Behavior

When a school administrator first connects their data source to your application, Edlink performs an initial synchronization. Unless you have pre-configured Sharing Rules to filter data, this initial sync will import the entire dataset from the provider.

This action generates a created event for every single entity (every student, teacher, class, enrollment, etc.) that is imported. For a large district, this can result in millions of events being generated immediately upon connection.

Due to this high volume, processing the initial data load via the Events API is often inefficient. Instead, we strongly recommend performing an Initial Full Sync using the standard Graph API endpoints (e.g., /api/v2/graph/people, /api/v2/graph/classes) to populate your database.

Once your database is populated, you can store the ID of the most recent event and begin using the Events API for incremental updates (deltas).

Nightly Syncs

Many of our clients opt to perform this procedure on a nightly basis to see what's changed in the data source over the course of the day.

In order to do this, we'll paginate through the events endpoint to see what's changed since we last checked. The endpoint will return events in chronological order.

The format of event objects received into the processEvent function of this example code is described here.

// Create an axios request config with our token
// (Remember, it should never actually be hard-coded like this)
const config = {
    headers: {
        Authorization: 'Bearer XxYUqqrxHBo6yWBqcry8b73GhibnrQyq'
    }
};

// Should be set to the id of the last event you've processed
// We'll be looking at the events that occurred after this one
const after = '00000000-0000-0000-0000-000000000000';

// This field will hold the url for our next request
// We'll get 10000 items at a time for maximum efficiency
let url = `https://ed.link/api/v2/graph/events?$first=10000&$after=${after}`;

// While our url is not undefined or empty
while (url) {
    // Wait for the result of the api call to Edlink
    const result = await axios.get(url, config).then((res) => res.data);

    // Loop through the events contained in the response
    for (const event of result.$data) {
        // Write a function to process events as they come in
        await processEvent(event);

        // Make sure you record the id of the last event you've
        // processed, so that it can be set as the `after` const
        // next time this code runs.
    }

    // Set the url variable so we get the next page on the next loop
    // If $next is undefined, our pagination is over & the loop will end
    url = result.$next;
}

A more complete sample code file for performing a full sync can be found here.