Much has been written, discussed, consulted, and debated about the collection, storage, and retrieval of Big Data. We will not attempt to add anything further on this complex and multi-faceted topic; rather, let us focus on the next question – now what?
Drawing insights from this Big Data to achieve actionable business intelligence is the ultimate objective. However, with Big Data comes Big Challenges to meaningfully communicate and interpret the data. Data visualisation holds promise as one tool to facilitate this communication and interpretation. With well-functioning data collection, storage, retrieval, and visualisation, a firm should be able to generate meaningful insights to drive business objectives.
We are all familiar with data visualisation – pie charts, bar charts, scatter plots, amongst many other types. Most of us can even use a spreadsheet program to create examples using data on two axes and an example or two to guide us. Data visualisation on Big Data is similar but presents some unique opportunities and challenges for investment managers specifically and financial services generally. For example, while a picture is worth a thousand words, a thousand pictures can be a bit overwhelming.
One of these challenges is to identify the type of insights one can achieve with this data. Functional categories relating to firm activities would include: investment decision and analysis, trading performance, operational measurements, risk analysis and oversight, and compliance and regulatory information.
The second step is to consider each potential audience member. As data visualisation exists solely to better communicate information, the question of audience is key. Portfolio managers, traders, operations staff and management, and risk and compliance officers are the immediate audiences that come to mind. In addition, let us also consider Board members, investors, and regulators.
We now have a data set, categories for us to investigate, and various key stakeholders as consumers of our efforts. How should we approach the problem of effective visualisation?
To get started, a quick peek into the world of academia to see what the professionals have to say is in order. The noted statistician, Edward Tufte, professor emeritus at Yale University, provides nine guiding principles for graphical representation of data in his seminal book The Visual Display of Quantitative Information:
1. Show the data
2. Induce the viewer to think about the substance rather than about methodology, graphic design, the technology of graphic production or something else
3. Avoid distorting what the data have to say
4. Present many numbers in a small space
5. Make large data sets coherent
6. Encourage the eye to compare different pieces of data
7. Reveal the data at several levels of detail, from a broad overview to the fine structure
8. Serve a reasonably clear purpose: description, exploration, tabulation or decoration
9. Be closely integrated with the statistical and verbal descriptions of a data set
Ideas and brainstorming
To bring the above principles towards a pragmatic approach to a project, let us consider the following topics more fully to define the visualisations of most benefit.
1. Not just know but understand your audience. Spend time asking the ‘right’ questions; based upon your audience, this is the place where one should spend most of your time.
- What does the user do today?
- What reports or information do they already receive that could be improved or augmented?
- Would more timely information be a benefit?
- Do the users prefer numeric grids versus charts and graphs for certain types of data?
2. Consider ‘traditional’ visualisations
- Time series data in two dimensions – trade breaks, compliance breaches, VAR and related risk measures
- Trend and correlation identification and analysis – trade error rates that correlate to certain counterparties or dates (option expiry, futures rolls, etc.)
- Delta spotting – a spike in risk, trade breaks, etc.
- Comparisons – performance versus benchmarks or peer groups, measuring service providers
- Attribution – explaining investment performance both to investment team and investors
3. Advanced techniques
- Statistical and regression based analysis identify trends – statistical analysis can identify correlations that are non-obvious
- Holistic measures of data activity and data growth – metadata measures of the health of the entire operation
- Measurement of usage of the tools – capturing time spent by each type of user can drive more efficient design and optimisation of the tool sets.
Once you have designed your desired visualisations, a few topics remain to achieve success in your project.
- Pick appropriate technology – some visualisations can be supported within your existing applications, others may require specialised tools or facilities. With well-defined scope, pick appropriate technology – avoid the urge to be dazzled by fancy or over-the-top demonstrations.
- Partner with your service providers – your existing providers are typically well-suited to provide assistance with some or all of your needs.
- Design the solutions so that they CAN be implemented and used with your own organisation. Culture and politics matter.
- Measure, observe, and trail the visualisations throughout the development phase. This is new ground for many users – take your time and listen. Return on investment should be paramount in your project team’s mind. You are not building tools to look cool – but rather to improve business results.
We are entering an era of exciting data visualisation of Big Data. While the topic can seem overwhelming, by following proven guiding principles and managing an engaged user-driven project, your firm can benefit from quantitative and qualitative improvements to your results.
By Gary Brackenridge, Global Head, Linedata Hedge Funds