My GSOC Journey #4: The code, and beyond

6 min readAug 20, 2021

My GSOC’21 proposal was a two-part project, aimed at adding features to Agents.jl. The first proposal was adding the capability to save and load agent-based models to and from files. This enables models to be restarted from any point easily, and also enables transferring the entire model state to any other system. The second part of the proposal focused on improving path-finding functionality in Agents.jl. It improved flexibility by reworking the user API to allow granular control over pathfinding parameters, and support multiple different profiles for pathfinding. Additionally, it also added path-finding support to ContinuousSpace, allowing it to be used in a wider variety of scenarios.

The multiple pull-requests implementing this functionality have already been merged into the main repository, and released as new version(s) of the library. Throughout this process, my mentors, Tim DuBois and George Datseris, and I regularly communicated to ensure we were moving in the right direction. They constantly ensured my work was up to the mark, and I learned a great deal from them.

What is Agents.jl?

Agents.jl is an agent-based modeling library. As stated in the documentation it enables performant, flexible, and scalable agent-based models to be created easily. Agent-based models (ABMs in short) are used to simulate a variety of phenomena, from epidemic spread to chemical processes, in a bottom-up manner. The user creates independently acting entities (agents) whose behaviour is determined by a predefined set of rules. The agents interact with and respond to changes in their environment, including other agents. For an excellent further explanation of agent-based modeling, and the capabilities of Agents.jl, refer to Tim’s amazing talk at JuliaCon’21.

The project

All the pull-requests (PRs) involving my work and the issues opened during this period can be searched for in Agents.jl and InteractiveDynamics.jl using the GSOC tag as a filter.

Before the coding period: CSV Integration

My project started two weeks before the official coding period began. Tim, George and I decided that it would be good to get a head-start on the project while participating in The Julia Language’s excellent community interaction period.

We decided to add the capability of loading and saving agent properties to and from CSV files. This involved using CSV.jl to serialize and deserialize agent data. The relevant work can be found in this PR and the issue it closed. The result added two new functions: populate_from_csv! (to populate a model using agent data in a CSV file) and dump_to_csv (to dump agent data to a CSV file). These functions are designed to be flexible enough for a wide range of scenarios. They also make use of metaprogramming to reduce verbosity, automatically inferring several details about the data to be saved/loaded.

To know more about this part of my project, check out this blog post!

Phase 1: Model (De)Serialization

This phase involved implementing model (de)serialization, as in the proposal. The relevant work can be found in the following PRs:

as well as the following issues they closed:

We decided to use JLD2.jl for the serialization and deserialization functionality. It provided an easy to use format that can handle complex types and nested data structures, which served our purposed perfectly. Although JLD2.jl provides its own custom serialization functionality, it proved to be too limiting for our purposes. Models also contain some non-serializable data, notably functions, which needed to be provided during model reconstruction at deserialization time. To enable passing this data as keywords during deserialization, a simple type conversion functionality was added. It recursively converts all relevant types to their serializable versions and vice-versa. This functionality, along with the CSV integration mentioned above, is extensively documented and tested. Writing tests for model serialization (referred to in the documentation as “checkpointing”) proved to be a complicated endeavour since it had to account for a large variety of models and spaces. This, however, revealed several bugs that were resolved as soon as they were found.

As an additional feature, we also decided to use Scratch.jl for keeping track of update messages printed in the REPL. The feature was merged in this PR and closed this issue.

To know more about this phase of my project, check out this blog post!

Phase 2: Pathfinding

The final phase of the project was improving pathfinding and adding it to ContinuousSpace. The relevant work can be found in the following PRs:

and this issue.

This was by far the longest and toughest set of features to implement. After a long discussion, we decided to completely and breakingly rework the pathfinding API. This allowed the AStar struct holding all the relevant data to be entirely separate from the model, and also allow multiple different structs with different parameters (representing different profiles) to be used. The changes provide a framework for future additions to the pathfinding API, including new algorithms. Using this framework, pathfinding support was added to ContinuousSpace. This helped to test the extensibility and scalability of the framework. An example to document the new capabilities of the pathfinding system was also created. It went through several revisions, transitioning from 2D to 3D and from GridSpace to ContinuousSpace.

To know more about this phase of my project, check out this blog post!

Stretch goals

After completing my proposed changes, Tim, George and I decided I should work on some stretch goals to further improve the new features. These include:

Benchmarking and improving upon the performance of the A* pathfinding functionality. (issue) (PR)
Adding compression support to the checkpointing API. (issue)

Beyond the code

GSOC is not just about the code you write. It creates a platform for open-source enthusiasts to interact, both within and across organisations. I had the opportunity to meet a great number of people, working with different organizations, using different technologies, and with vastly different interests.

During the community bonding period, The Julia Language created a slack channel for all of us to interact. They also created a custom gather.town, where we could all meet as virtual avatars of ourselves. Several of us used this extensively to talk about each others’ projects, and more. Meeting people from all over the world, working on all sorts of projects, was a great learning experience.

Most importantly, though, I would really like to thank my mentors Tim and George for being the best mentors I could have asked for. Both of them were extremely responsive with all my ideas, issues and changes. They considered every idea (and there were a lot), and were very helpful with every problem I faced, large or small. I also value that they took time out of their busy schedules for our weekly video calls, discussing the progress made and the plan for the next week. They were very prompt in their feedback to any changes made, constantly providing helpful reviews. Both of them also provided interesting and useful insights about their own work, the Julia ecosystem as a whole, and various aspects of academia.

I am extremely grateful that they showed confidence in my abilities to find solutions to difficult problems. Tim and George involved me in matters even beyond the scope of my project, including things such as the direction of Agents.jl as a whole moving forward and the talk at JuliaCon’21.

I will definitely continue to contribute to the Julia community in the future. I look forward to working more on Agents.jl alongside Tim and George, to make it the premier agent-based modeling library (if somehow it isn’t already).