Building a Web-Based Tool to Interact With Geospatial Data
Case study coauthored by: Emma Ungureanu, Andrei Ismail, and Abhishek Maity (Fraym)
Fraym is a geospatial data company that brings innovation into decision-making processes of governments and private organizations. By using machine learning to produce local-level consumer data, Fraym adds new dimensions to the way data is gathered and processed in emerging markets.
Fraym needed to make data available to customers in a centralised, visually powerful, customizable way. To achieve this goal, Fraym partnered with Vitamin Software to build a web-based tool that allows users to interact with geospatial data.
The dataFraym web application enables public and commercial entities to access granular information on the economic, behavioral, and demographic characteristics of communities. The resulting application helps strengthen the relationship with new and existing customers, and provides Fraym with a recurring revenue stream.
About Fraym
Fraym is an innovator in the field of data science for revolutionizing data-driven decision-making. Recognized by FastCompany World Changing Ideas 2020, Fraym uses machine learning models to generate hyper-local insights on communities around the globe. Fraym's location-based data about population characteristics and behaviors are employed by governments and organizations tackling some of the world's most pressing challenges.
Using proprietary algorithms, Fraym produces datasets that cover a wide range of indicators, including socio-economic, attitudinal, media consumption, health, education, and access to services information.
This advanced geospatial data and analysis – available for 100+ countries at one square kilometer resolution – is delivered through dataFraym, a web-based platform that allows users to generate customized reports and answer strategic questions.
“Fraym was built to help fast-growing companies, development organizations, and governments succeed in places where data has been traditionally hard to access.”
— fraym.io
Customer Challenge
Fraym uses advanced machine learning models to generate unique insights by weaving together primary, remote sensing, and satellite imagery data. The output is foundational data that adds an entirely new dimension to analysis and reporting. Customers – whether private companies or government institutions – engage Fraym to obtain specific, custom-tailored data to augment their decision-making process.
The main challenge for Fraym was how to present data to its customers in a consolidated and user-friendly interface. Before we joined forces, Fraym’s data science teams were producing bespoke solutions with static data narratives and limited visual representations. This approach limited clients’ ability to explore and interact with the data, which was limiting the reporting and analysis potential.
Furthermore, other existing SaaS providers were not able to accommodate the rendering of raster data at the 1x1 kilometer resolution of Fraym’s data.
Our goal was to build one single product that could bring together all the bits and pieces: a dynamic web platform that all customers could access to meet their data needs. Fraym’s specific technology requirements were:
- A user-friendly interface that is easily adjustable to address customer requirements
- Resizable compute capacity to handle an ever-increasing amount of data
- An information system that supports viewing, editing, and analysing geospatial data
- A secure database system to store proprietary data
Fraym leveraged Vitamin Software’s expertise to choose the right technology stack and to identify a sensible prototype that would allow clients to access tailored information.
Why Fraym Chose Vitamin
We at Vitamin deliver value fast. Yes, it’s that simple! Vitamin Software immediately demonstrated an ability to help Fraym work more efficiently to build the product they envisioned. Of course, speed is not the only advantage we bring to the table, but our joint capability to move fast in the same direction definitely helped Vitamin and Fraym “tie the knot”.
"[…] I'm so glad we partnered with Vitamin. Their work is excellent (and fast).
But more importantly, they've helped us see blind spots, consider alternative approaches, incorporate new product learning, and think through new strategies. We've made tremendous progress with Vitamin's help, and we look forward to more great progress ahead."
— Dustin Homer, Director of Solutions, Fraym
There were many qualities that made our relationship with Fraym so successful, but Vitamin Software’s main differentiator in the long-term is an ability to work with customers beyond the minimum-viable product (MVP) stage. By leveraging technical and communication skills, we first built a strong prototype that met Fraym’s needs. We then nurtured the product until it reached maturity, helping iterate and operate dataFraym until we achieved product-market fit. And today, we continue to help Fraym further develop the product to match their evolving business needs.
Fraym chose Vitamin Software specifically for our well-rounded skills in multiple areas: front-end and back-end development, infrastructure setup, project and product management. And because we are product-focused, we are capable of navigating the murky waters of finding market fit.
The Solution: Putting Data into dataFraym
Fraym’s essential asset is geospatial data. The company gathers data from various sources – made available by governments and organizations, or primary data gathered from surveys and studies – and puts it all together, using machine learning-driven modelling to derive complex indicators. The question Vitamin Software had to answer was how to visualize Fraym’s data and bring valuable insights to Fraym customers in an efficient, user-friendly, and customizable way.
Vitamin Software suggested to first create different prototypes in order to understand the needs of end users, and develop requirements for the product Fraym wanted to build. Implementing incremental improvements to the prototypes, we finally found the answer: the dataFraym web application, built from scratch and now a pillar of the Fraym brand.
Screenshot of dataFraym: Population density and socio-economic indicators for a 1x1 square km area of Abuja, Nigeria
Our standard practice of delivering code changes frequently and reliably – of continuous delivery – combined with our experience with Amazon Web Services (AWS) served as the starting point. We at Vitamin Software set up a secure infrastructure where we shipped to production weekly, using a reliable technology stack:
- Travis CI, a hosted continuous integration service is used to build and test the software we create for Fraym
- AWS is leveraged for secure data storage, processing and monitoring
-
- Amazon Elastic Container Registry (ECR) and Amazon Elastic Container Service (ECS) are employed to store, manage, share, and deploy the Docker container images
- The dataFraym application connects to a PostgreSQL database hosted within Amazon Relational Database Service (RDS)
- The data shown in the application is hosted within CartoDB, an open source geo-data management platform; due to limitations of CARTO’s hosting service, we decided to house the data ourselves on a dedicated Amazon EC2 machine with a standard CartoDB deployment
- An AWS Load Balancer is used to ensure the security of the application, and for scalability when needed
- Amazon S3 buckets are used to store static resources for the application (images, icons, Load Balancer logs etc)
- Amazon CloudWatch is leveraged for monitoring by setting up alerts and tracking logs
- Amazon Simple Email Service (SES) and Amazon Simple Notification Service (SNS) are employed for communication with dataFraym’s customers
- Intercom is used to ensure an outstanding customer service and support
- Python and Django are leveraged for the application back-end and API; the API is modeled through a Django REST Framework
- Testing is implemented via Selenium
- Error monitoring through Sentry complements the deployment
- Angular is used for the application front-end; we use a redux-like pattern for client-side data storage and processing, and Mapbox and CARTO.js client side libraries for geodata display.
Coming Out of the Woods
Like any other project, we encountered difficulties along the way. The first challenge we had to overcome with dataFraym was related to geospatial data storage and retrieval. We needed a system that would allow fast retrieval for geodata via http requests, in a format ready to display via client-side libraries like Mapbox. To solve this, we used the CartoDB system with a proprietary application layer to translate REST-style requests to CartoDB SQL queries.
Once data storage and retrieval was in place, the next challenge was user-friendly data presentation. This was achieved with Mapbox and CARTO.js; the client side libraries allowed a high degree of customization for map data visualisation. Finally, users need to control what is displayed on the map with a high degree of accuracy. To build the UX, we relied on Angular’s efficient componentization and data access patterns.
How Everything Ties Together
So we knew what we needed to build the product, but how did we deliver such a successful solution? Leveraging our communication skills, Vitamin Software’s model is organised as a one-stop-shop for the customer. Fraym works with a single Vitamin Point of Contact – the Project Manager – who has a close relationship with the Fraym Product Manager. The Project Manager attends regular calls to gain insight into the Fraym’s initiatives, and leads the internal Vitamin Software project team to deliver incremental results at a steady rate.
Results & Benefits
The goldmine for any company is a product that brings recurring revenue. The dataFraym web application is now a key Fraym asset that does just that. The system we built together is not just software – it’s a product with a subscription-based pricing model. Having used our business intuition, we were able to understand Fraym’s end users so well that they stick around after trying out the application.
Numbers Speak Louder Than Words
dataFraym is a valuable tool that Fraym also uses for customer development. Apart from the revenue it generates, the app is used to demo the many functionalities to potential customers. It was also used to run a successful beta campaign to inform pipeline development, gaining valuable understanding of sales leads. The results are obvious, as Fraym’s revenue has grown substantially over the past two years: direct dataFraym sales revenue doubled between 2019 and 2020, and the number of countries supported also doubled.
The Icing on the 🥕 Cake
Working with a business-savvy partner like Vitamin Software, Fraym did not have hidden costs incurred by poor communication and lack of business intuition. Apart from the direct value of the application, Fraym has benefited from the collaboration with Vitamin Software through the following initiatives:
- An API was built to help Fraym’s customers incorporate the data into their own workflows smoothly; organisations are able to pull in data from dataFraym automatically into their own tools and integrate the data into their own processes
- Fraym’s operational processes have been streamlined to ensure a smooth workflow between internal teams
- With structured, streamlined processes in place, we were able to set up a Learning & Development program where Vitamin helps Fraym build their internal capacity for technical work
- Having a robust, secure, and stable system in place, Fraym is able to establish its long term vision and scale their activity
Next Steps
Fraym is an innovator in the field of data-driven decision making, and Vitamin Software is a business-savvy partner who translates the innovations into concrete functionalities for end-users. If you think of dataFraym as a car, Vitamin Software provides both a repair shop to keep the car in good shape and a driver that takes Fraym where they wish to go. We plan to support Fraym in its start-up to scale-up transition, and our top priority along the way is to ensure that data produced using proprietary algorithms is secure.
Secondly, Vitamin is working on making the system scalable, so that it can accommodate the ever-increasing amount of data that Fraym collects, processes, and stores. To achieve this, we are already working together on optimising operations, like simplifying the process to onboard a new customer and decreasing the time to add data for a new country by automating more and more steps.
Last but not least, we support Fraym in improving operational processes for transforming data, making sure these processes are efficient and keep up with Fraym’s growth.
About Vitamin Software
An outsourced product development firm, Vitamin Software specializes in building your product and taking it from MVP to product-market fit. The customer leads and we drive the product to follow.
Vitamin Software is an Amazon Web Services (AWS) Partner Network Select Consulting Partner, specializing in building, operating, and maintaining robust and secure products. We operate the production setup in a secure fashion, based on our AWS Competency.
Vitamin Software builds bridges: From problem to solution, from difficult to feasible, from impossible to outstanding. Want to know how?