Thirst for detailed knowledge of the modern industries, up-to-date and accurate information on general world issues, inspired the Webster Pacific LLC to create an open data-source platform that will allow professionals of a wide range of interests to exchange and share useful data conveniently and effectively. To realize this idea and to make any helpful and relevant information available from any Internet-connected PC on a regular basis, the company appealed to thorough assistance of our team.
The business goal of the project was to create a web application able to host statistics on important cultural, economic, social and environmental issues. The primary functionality of the software was to allow visitors to view and download data and make it possible for users to input, update, view, report on, organize, share and download data on line.
All information presented on the website should be comprised of different data categories – platforms, representing the public or proprietary data of client organizations, subscribed to them. Idea behind was to provide the potential users the possibility to create within the platform either a public site, which is free of cost, or a private site, which has an associated cost.
One of the major challenges was to develop a high performance web application with complex functionalities, which presupposed:
- Data storage module to improve the overall cohesion in the data structure, by relating table records to nodes, adjusting data query and search.
- Data visualization module to provide the contextual visuals, including clustered data sets, diagrams, plots, charts and dynamic data graphs.
- Data feed module. Generalization of data feeds by using ADF (Automated Data Feed), which helps to load data from external sources automatically.
As mentioned above, the project has several integrated platforms, all consisting of miscellaneous reports, which in their turn are comprised of a series of "graph groups" (web pages with 1 or more graphs) containing enormous data behind them. One of the massive challenges was to optimize the data storage and processing and thus to harness the growing capacity of on and on uploading information, which required more reliability and universality in the data structure itself.
Our working team came up with the solutions that were extremely useful and efficient with following features:
- Optimization of site's performance, which helped on the current stage of the project to overcome difficulties with activating multiple data sets simultaneously, that as a result made data reviews less time consuming.
- Some extremely powerful data-visualization tools were integrated to create data reports, which dramatically increase the range of possibilities for using data in visual storytelling.
- Introduction of the site's organizational tools that saves time and energy of user to upload and review data. As an example, our team integrated “easy upload” module to assist subscriber in placing his data on site. The user can choose between pasting data from another source or typing it in, or uploading data from an Excel CSV file. One more effective option is the implementation of the ADF - a tool to load data from external data sources (other web-sites for instance) by schedule or manually.
- Providing of the clear context to made reports logical, organized, graphical and insightfully commented. To make the site's appearance user friendly and eye – catchy, the contextual visuals were thoroughly considered and implemented in final UI variant.
- Improvement of data processing by introducing specific data storage patterns – Data Plots, which gives more flexibility for data handling.
All the implementation stages were completed in a close collaboration with Webster Pacific team to insure that all the provided solutions align with the costumers highest expectations. Beta version was launched and fully tested on the preproduction server and moved to the production server after the customer's approval.
Client is successfully using this web-based solution to provide financial and operating reports for their numerous clients by using information provided by the world’s leading data publishers, available on site.
Data360 serves as an additional and important tool for collaborative market analysis. Client gets an advantage of using thousands of data series amassed on the web-based application and explores possibilities of its further practical application.
Further development plans include dealing with growing site's capacity and finding a way how to effectively import and export data from/to different file formats and data sources, which requires refactoring of newly uploaded data to the acceptable and convenient format.