Business Intelligence & Data Warehouse Practice

Historical Perspective

From the creation of the relational data model developed by Dr. Codd through the evolution of computing technology and relational database engines popularized by such companies as Oracle and IBM, people have been trying to make sense out of the ever increasing amounts and types of information available. As data storage technology and hardware became faster and less expensive, companies began to find use in storing and analyzing larger and larger volumes of data. Where storage of a few months of sales data was adequate and much of the analysis across great timeframes was done manually or avoided completely, managers began to demand that more information and various types of information be stored for quick retrieval. Comparing the previous period’s sales to historical periods now needed to be done quickly through one interface. Moore’s law seemed to be well in place during the 80′s and 90′s with data storage and retrieval capacity growing at phenomenal rates. Yet, the ability to quickly retrieve and display large volumes of data in a useful, intuitive manner which aided business decision processes was still waning. Ralph Kimball introduced dimensional modeling and the term “Data Warehousing” (DW) in the early 90′s which provided a framework for storing data in a manner designed for quick retrieval. Companies such as IBM (Cognos) and SAP (Business Objects) capitalized on this technology and produced interactive query tools laying the groundwork for efficient retrieval and display of larger volumes of data. The term “Business Intelligence” (BI) evolved during this time as companies realized this intuitive access to large volumes of data could assist them in making intelligent decisions about their business processes. Still, challenges persisted as demand for simpler interfaces with greater capabilities continued.

Today’s Challenges

Several dominant players exist in today’s BI marketplace. Acquisitions and mergers have proven that only vendors who can provide a complete line of DW and BI products will survive the competition. Traditionally, reporting or analysis based companies have acquired smaller, niche companies to fill in their technology gaps. Database vendors such as Oracle, IBM and Microsoft have developed or acquired reporting tools in an effort to compete in the ever increasing BI marketplace. Most recently companies have incorporated Hadoop capabilities to augment their BI functionality to include non-relational or “unstructured” data sources such as web logs. Additionally visualization tools such as Tableau and Tibco Spotfire have pushed analysts to consider unique visualization methods to identify data trends or outliers. Today’s BI challenges are centered on the following areas:

  • Data visualization – How to identify a business problem or opportunity using visual data presentation methods.
  • Data extraction and aggregation – How to retrieve the data from the database, files or unstructured data sources and summarize it to a useful, presentation level quickly.
  • Scalability – Larger numbers of users and larger volumes of data.
  • Interoperability – The tool must interface with many databases and emerging interface protocols.
  • Interface – It must be available on mobile devices and web browsers with similar functionality as a traditional desktop based tool.
  • Big Data – Today’s businesses are able to collect increasingly large volumes, variety and velocity (the 3 V’s) of data and incorporate it with semi-structured data seamlessly in one integrated platform.
  • Document Databases – The increasing popularity of document-oriented and NoSQL databases further complicate BI needs as data is stored in a combination of semi-structured and traditional data stores.
  • Agile BI – The need to extract, integrate and present data quickly during BI application development to facilitate business input and code modification before promoting code to production environments.
  • Business process integration – BI tools must support or enhance the existing business processes.

The last point represents one of the greatest challenges in today’s marketplace. Failure in this area is typically the result of incorrect or inappropriate application of technology, a lack of understanding of the business process or problem, or inadequately trained personnel during the design, development, and installation phase of the project. Whereas the challenges of the past may have been storing the data in a method configured for retrieval; today’s challenges are centered on viewing the information stored in the data in a way to support efficient business decision making processes.

Tomorrow’s BI

Industry leading IT research companies see the merging of BI companies continuing for the foreseeable future. As today’s larger BI vendors are struggling to keep up with the functionality offered by their competitors, they will likely continue to acquire niche vendors to provide a complete enterprise Business Intelligence suite of products. The challenge of these BI vendors will be to provide a consistent message and support their existing customer base while integrating their new product acquisitions. Users of BI tools need to hold their vendors accountable to the following criteria:

  • Support of existing customer base – Is the vendor giving this appropriate priority?
  • Strategic alliances – Is the vendor aligning to other companies whose products are used elsewhere in the corporation? Do these alliances fit into our corporate infrastructure standards?
  • Product direction – Are the mergers, acquisitions and new versions creating functionality which is essential in our corporation? Are our product needs a priority within the vendor as the product line grows and changes through mergers and acquisitions? Are products bundled in a fashion that is advantageous to our current and future usage needs and requirements?
  • Corporate and financial stability – Will the company and their intellectual capital still exist when we are using their products several years from now?
  • Is this a fad? Is the vendor applying their latest technology to my business merely because it is a fad? Is there truly a need for this technology within my organization that will enhance and support my business processes in the future?

Predicting the future of BI vendors and the use of their tools may seem insurmountable and may not provide a reasonable return on investment given today’s ever changing market conditions and evolving user requirements. However, it is important that we attempt to align with BI companies whose future appears solid to form a critical segment in the stable foundation of a corporate BI infrastructure.

Why Pinnacle?

It is the opinion of Pinnacle Solutions Group that success or failure of a project is not guaranteed by tool selection, vendor selection, or technology selection. On the contrary, project success is guaranteed by the integration of experienced DW/BI personnel, working together as a team from the conceptual phase through implementation and support with a priority toward solving the business problem through the use of appropriate technology. Pinnacle Solutions Group strives to maintain a balance between the application of technology and the business need. They are stakeholders in their client’s success, solving the business needs through the use of technology, an understanding of the business problem and effective communication skills. Pinnacle Solutions Group believes that experienced DW/BI personnel demonstrating all of these capabilities can deliver appropriate BI solutions that insure their client’s vision is realized.