Wednesday, March 30, 2016

Presentation and Visualization Methods

Presentation and Visualization Methods


The era of today’s technology and corporate culture has a lot to deal in terms of the data that needs to be analyzed. In addition, time constraints are applicable on almost all the professionals. This calls for coming out of the conventional way of analyzing figures and numbers and stepping in for visual depictions that makes the interpretation of events easier and quicker. The specific visual, however may be good for some and otherwise for others. The intended audience and their requirements need to be paid attention to first. There exists a lot of effective presentation and visualization methods for various business vignettes such as telecom bills, insurance premiums, order summaries etc. Though almost every technique can be utilized to represent any of these, there exists special visuals that outweigh any others with respect to the appeal and easy comprehension by the audience who views it. Whereas bar charts may be sufficient to view the time trend of a particular quantity (ex: demand for a product), other visuals such as heat maps, scatter plots, dashboards are equally important in other domains of relevance.
We choose the following business vignettes to describe the variations of visuals that may be used to represent the relevant information to the intended audience.
·       Retail sales
·       Financial Services
·       Insurance

   For each of the vignettes mentioned above, the following content elaborates upon the requirements expected out of each of them followed by the recommendations on best visuals and sample depictions.

1.    Retail Sales:


The audience for viewing reports on retail sales varies by size of the organization as well as role of the people in respect to the organizational hierarchy. Big organizations’ management desires a global report with a granularity of region/continent level. People lower in the hierarchy need it to be more granular in terms of both timeline and regional breakup. The sample chart depicts the markdown in terms of store and product category at a specific regional level. The report further highlights the absolute pound amount on right hand side. Along with the bar charts, outliers are highlighted in the charts that reflect on aggregate measures, ones that may be of interest to the management in the organization.

2.    Financial Services:



The financial organizations such as stock exchanges, bullion exchange need to analyze the data from a multitude of angles. Whereas some may like to monitor stock prices of a company over time, others might want to explore the trend in overall market index. Currency monitoring is another facet where major currencies of the world are monitored on price relative to US dollar amount. The sample cited above uses line graph to monitor the currency prices over time. The report also has a zoom-in feature to dig into granularity of hours and minutes in a day. Scatter plot depicted on the right depicts the trend relative to the percentage volatility in a currency for a particular period of time. This kind of report is routinely used in institutions such as stock broking firms, equity banking firms etc.

3.    Insurance:



The management in automotive insurance companies are interested in improvising the probability model upon which they charge premium on various products offered. On these lines, they often wish to analyze time series data of the number of accidents happening on a yearly basis compared to the premium they charge in a particular year. They are also interested in monitoring the claim ratio of the vehicles involved in order to offer competitive premium charges to future customers. The sample cited depicts the number of accidents and claim ratio in a bar chart format with plotting lines depicting the other two variables. Such a model suffices in easy comprehension of the data over the years which in some way also helps analyze performance of the company.

The above mentioned depictions and explanations are brief overviews on types of visuals and presentations desired by different kinds of organizations and stakeholders. They are not exhaustive and highlight the features that are relevant to the samples that have been cited along with them.

References:

Tuesday, March 1, 2016

Structured & Unstructured Data/ Data Warehouse

Structured & Unstructured Data/ DW

Recent advances in the field of business intelligence and analytics definitely calls for the research and analysis of the highly complex domain of unstructured data as compared to the conventional forms of structured data. To start the discussion, it will be great to define these two disparate yet integrated domains of data for the purpose of laying down the foundation for this discussion.


Structured data:
The most common form of data that is organized in the form of tables (Rows and Columns), has well-defined data types (Alpha, numeric, alphanumeric) without any ambiguities in precision. This form of data is quite easy to be loaded into analytic tools without or with very little pre-processing. The analysis with such a kind of data is quite easy in terms of effort required. The data here is ready to be understood by the computer/machine.
Irony here is that though this has been used over decades, yet it consists of only about 20% of data that is available to develop valuable business insights.

Unstructured Data:
This form of data can be considered as nearly opposite to what structured data is. It consists of unorganized data that is void of well- defined data or data types ready to be comprehended by machine/ computers. About 80% of valuable data is composed of this type. The most common examples can be emails, word documents, PDF, social media data etc. A lot of effort is required to be put in terms of extract, transform, and load to make this data ready for analysis. It can be considered as data that is readily human comprehensible but not machine ready. Recent developments in technology have opened up this unexplored domain for valuable analysis.
Summary:

·      Present state of data and data warehousing for analysis

The ‘industrial revolution of data’ as they coin it, is what the organizations are witnessing globally today. Moreover, the proactive ones have already started putting their resources to work in an effort to gain the maximum out of it. The Big Data includes all forms of data ranging from unstructured, semi-structured to structured data. The development of tools and techniques is in parallel with the unveiling of the exabytes of data, storage techniques are facing a challenge to cope up with the times. As an example, the famous retail giant Walmart handles more than one million customer transactions on hourly basis that are fed into the databases storing more than 2.5 petabytes. Other technology endeavors such as IoT, Cloud implementation, security enhancements are complimenting this abundance of data today. Yet, as per the industry experts, this is just the beginning of an era that is yet to witness full- blown caliber of analytics to assist in decision- making for organizations.


Without any doubts, Data Warehousing is the perfect answer to the analysis of the kind of data available today. The fundamentals of the concept still remain unaltered, however development and research is a pervasive process to leverage the powerful abilities of DW with other technology advancements. There are tools available today that can harness data from even unstructured data based on a set of rules/ logic and fed into the DW. A key example for today’s state of DW can be witnessed from the way large retail giants like Amazon are able to offer customized shopping recommendations from the browsing pattern of their customers. The elements of dynamism and real- time data are the gold coins, DW has the ability to dig them out in an efficient manner.

·      Limitations of DW for different data types:
DW might have evolved to a great extent as far as the structured data is concerned but it still has limitations when it comes to unstructured data- one with more potential. One of the key concerns for analysis of unstructured data from a DW is the cost/ benefit ratio given the enormous resources and effort required to pre-process data to make it DW ready. Not only the processing, but the ballooning effect in the databases on account of massive amounts of data to be handled is another concern. Managers are unaware of the results to expect after putting in the massive amount of effort in handling these kinds of data and whether or not this will enable the profit drive for their organizations. The latency on account of pre-processing of data might hinder the ultimate objective of real time analytics. A lot needs to be done to leverage the data warehousing capabilities with the raw data available to gain full- blown advantages and this is a pattern witnessed in almost all the technologies built so far.

·      Future Trends in DW:

(1)    More Capable Data Warehouses
The burgeoning of data volume and types calls for fine-tuned DW’s that are versatile enough to handle these advancements. With memory requirements increasing on the warehouses, cost factor would need to be watched out for. A proportionate increase in cost of new system with increase in memory requirements does not justify well. Capability also needs to be enhanced in the areas of over-the-cloud deployment, mobile technology enabled systems that have a location independent appeal to the most avid viewers who are the managers of organizations to get a sneak- peek whenever, wherever required.

(2)    Physical and logical consolidation for cost control
As discussed above, to control the ballooning effect in cost, systems would require more of logical than of physical enhancements. The logical enhancements might come in the form of virtual systems, tightly synced databases that have the ability to reflect the real-time situations without any significant latencies in order to gain the most out of the data warehouses. A significant effort needs to be put in this regard.

(3)    Real- time analytics
The huge warehouses need to have the caliber to give insights to real-time updates from the system. Time has always been an invaluable resource to any organization, given the shrinking nature of window period for accommodating latencies in analysis and prediction today, efforts towards reduction of processing times from the system would need to be pushed in an integrated manner.

(4)    Cloud Integration
The developments in the cloud technology and its allied advantages call for a complimenting effect from the data warehousing front as well. Given the advantages of cost reduction and mobile access to data from anywhere, next step for DW’s is definitely directed in this direction. Moreover, the fundamentals of data warehousing have the most significant advantages aligned from this horizon. Leveraging cloud with DW is both a win-win situation and the demand of the times.

(5)    Beyond dashboards and reports- integrating day-to-day activities
The dashboard and reporting abilities from the DW that are instrumental in strategy formulation for driving the organization towards achievement of business objectives are far too limited to harness the full potential of the DW’s. Real advantages lie in the promising value it has in guiding everyday work of the organizations today. Things as small as budgeting of cafeterias in the organization would be tackled both qualitatively and quantitatively if the data is fed in appropriate manner.

References: