Roadmap¶

OCP prepares a software roadmap each fiscal year. This page documents common concepts for all roadmaps.

Product and service families¶

Application
- Credere: An online service that facilitates the participation of Micro, Small, and Medium businesses (MSMEs) in the Colombian public procurement market.
Use
- Data Registry: An online tool to find and access public procurement datasets that are available in the OCDS format.
- Cardinal: A command-line tool to calculate red flags and procurement indicators from OCDS data.
- Redash: A deployment of Redash to give users access to the Kingfisher database.
Evaluation
- Data Review Tool: An online tool to report structural errors in OCDS data. It is primarily used by OCDS implementers to check their conformance to the standard. Its library is used by Kingfisher to check for structural errors.
- Pelican: An online tool to report quality issues in OCDS data, sourced from Kingfisher. It is used internally by data support managers to provide feedback to OCDS publishers, and by program managers to better understand a publisher’s data.
Pre-processing
- Kingfisher: A family of tools to collect, pre-process, summarize and query OCDS data. It is used internally by data support managers to provide feedback to OCDS publishers, and by data analysts to calculate indicators and do research. It serves as a data source for other tools.
- OCDS Kit: A suite of command-line tools and a Python library for working with OCDS data. It contains common functionality used by Kingfisher, for example.
- Spoonbill: A command-line tool to transform OCDS data to tabular formats. It is an important tool for users who are more familiar with tabular formats.
Publication
- OCDS Documentation: The official documentation of the Open Contracting Data Standard.

Together, these tools connect with different steps of the hypothesis for change expressed in the Scope of Data Products and Services.

Health¶

For each product and service, this spreadsheet briefly describes its use, development status, future work and code quality, in order to give a high-level overview of its present status and future plans.

Principles¶

This section serves to clarify important considerations, not establish a comprehensive philosophy.

Understand user needs¶

It is of course of primary importance that our tools respond to user needs. See our development guide for new software for how user research is integrated into our approach. We also:

Listen to users and document their needs, when they communicate through the mailing list, GitHub repositories, data support, community calls, in-person events, etc.
Ask users about their needs, whether through the channels above, online surveys, phone interviews, or other methods
Observe users interact with our tools, whether indirectly through web analytics or directly through usability testing
Reflect on what was heard and observed, whether during OCDS retreats, team meetings, CRM issues, GitHub repositories, etc.
Promote the use of our tools and encourage feedback through the mailing list, GitHub repositories, and data support
Recognize that our tools are only one part of a user’s workflow and of a wider ecosystem
Transfer knowledge from relevant domains, e.g. open data more broadly

Create products sustainably¶

It is important to avoid both vendor lock-in and personnel lock-in (i.e. cases in which only one party has the knowledge and access to modify a product). To mitigate this risk, our approach is to:

Not build a new tool if, with reasonable effort, a third-party tool can be used, adapted or improved to meet our needs (e.g. jq).
Use a Quality Assurance Surveillance Plan (QASP) with consultants, which includes:
- Have test coverage above 90%, so that new developers can get involved without being overly concerned about breaking things.
- Have good documentation for developers and users, making it easier to onboard new developers. (Writing documentation during development helps identify and reduce complexity.)
Maintain this software development handbook, which includes:
- Use popular libraries where possible (e.g. Click, Django, Scrapy), so that developers have fewer APIs to learn.
- Use a small number of languages (primarily Python) and dependencies (like Django), so that it is less effort to update dependencies with security fixes.
Break big products (that change frequently) into small libraries (that change infrequently), so that a new developer isn’t required to know as much to work on a product.
Follow good practices so that code isn’t more complex than needed: YAGNI, KISS, DRY (refactor regularly), Unix philosophy (limited responsibility, loose coupling), proper encapsulation, etc.

Note

However, don’t sacrifice performance, maintainability or readability. See also Code, Object-oriented style and “Clean” Code, Horrible Performance.
Use the latest versions of languages and dependencies, to postpone costly upgrades as far into the future as possible.
Use tools properly and as intended. Don’t go for the quick fix.
Anticipate needs with respect to performance and extensibility.

The above broadly follows the spirit of Choose Boring Technology, which applies Maslow’s hierarchy in the context of software development. We try to ensure basic needs are always satisfied (Is it tested? Is there a maintenance plan?), so that we can spend more time on the big picture (Is it achieving user goals? Is it achieving organizational goals?).

Note

Our approach to refactoring is similar to that expressed in these posts:

Our approach to architecture is informed by:

Manage products and services responsibly¶

Among other practices, we:

Sufficiently resource the maintenance of all software, to manage technical debt
Sufficiently resource the hosting of all services, to avoid service disruptions
Review the health of products and services in each roadmap and plan maintenance and improvements accordingly

See our development guides for new software and simple websites.

Build capacity through documentation¶

As the users of our products and services grow more numerous and diverse, we need to ensure that our documentation meets different users’ different needs. See our approach to Documentation.

Collaborate with others¶

If OCP is creating and funding all the tools, its strategy is not working. We limit our software development activity to the highest priority tools in order to bootstrap data use and support internal operations, and otherwise encourage others to develop tools.

Outside our software development activity, we:

Promote others’ tools through our communications channels, like in blog posts.
Offer a mailing list through which others can share and promote their tools.
Suggest others’ tools, where appropriate, through data support.
Monitor reuse of tools authored by others as an organizational indicator.
Have bilateral calls with similar organizations about software development.

Within our software development activity, we:

Contract teams to deliver our major projects, to build capacity for OCDS tool-building. For example, Datlab was awarded the contract to develop Pelican (FY20).
Share new versions of our tools and libraries with other teams through the mailing list, so that they are aware of any changes.
Follow semantic versioning, so that others’ software can reliably use our libraries as dependencies.
Engage with other teams to explore collaboration on open-source tools. For example, we made JSCC (FY20) and OCDS Babel (FY19) reusable by other standards.

We also support others through time-bound projects. For example, in FY20, we worked with the World Bank Group and The Engine Room to author a primer on tool reuse in open contracting. In FY19, we funded tools supporting a Latin American journalists network. In FY18, we created the OCDS Bounty Program to support tool-builders to better document and package their existing OCDS tools for reuse. In FY17, we invited developers to participate in a prize-based Open Contracting Innovation Challenge, to support six innovative and experimental tools.