Andrew Wiles Olap Warehousing Blog: Mdx Query Builder

Multi-Dimensional Spreadsheet

Fri, 16 May 2008 07:54:45 GMT

In my last blog entry I described the work we have been carrying out on data entry in Intelligencia.

In parallel with this activity we have also added that ability to update the MDX script associated with a cube allowing users to add/edit cube calculated membes and cell calculations. This functionality relies on AMO to read the cube script and then allows the user to interact with the calculations via our spreadsheet calculation interface. At present this functionality is only appropriate for users with administrative rights due to the way in which SSAS handles security.

The next stage in the development process will enable interaction with write enabled dimensions allowing users to add/remove data bearing dimension members.

Once this work is complete we should be quite close to providing a modeling interface that is not dissimilar to a multi-dimensional spreadsheet. It would not be a far stretch to allow users to create new cubes and add dimensions/hierarchies allowing ad-hoc building of small OLAP models. An interesting challenge would be to allow business end-users to add to a corporate planning/budgeting application so that they can model the specific behaviour of their part of the business.

Data Entry

Fri, 11 Apr 2008 08:27:01 GMT

Over the last couple of months I have been working on some exciting new functionality for Intelligencia. This is the addition of data entry and modeling capability to the query builder. The ultimate objective of this work is to provide a comprehensive user interface in which an end-user can build an Olap data model and populate it with data. The first phase of this work concentrates on providing management tolls that sit on top of an existing data model.

Data Entry

The data entry component of the work allows interaction with write-enable cubes. The challenge here is to provide an intuitive means of handling allocation of data from aggregate cells. What we have decided on is to only allow data entry where a member is an aggregate in one dimension only. This restriction mens that it is clear on the screen where the data is going to be spread and also limits th possibility of creating allocations to huge numbers of cells. In the screen shot below the cells highlighted in orange can be used for allocation and thise which are gray cannot (and are also protected).

The next challenge was to come up with allocation rules that behave well in different circumstances. The first thing we encountered is that for parent-child dimensions data will be allocated to the data member for an aggregate node even if it is not displayed. The result of this is that the user enters a number into an aggregate node and finds the sum of the children when allocated does not appear to equal the total. We have therefore had to create allocation rules which prevents data from being allocated into data members. We have also made a distinction between allocation when the aggregate member has no data (when initializing the model) and allocating to a pre-populated model. In the first case we create an equal allocation in which all child nodes (except the data members) receive the same data value. Where values already exists we carry out a weighted allocation where each node receives a value in proportion to its contribution in the original total. The result of this work is intuitive and does not require any unerstanding of MDX or SSAS by the end user.

To top up the default allocations we have created a user interface for spreading data into ranges of cells based on linear increases/decreases in value and seasonal profiles. The following screen shot shows the seasonal profile window in which a user can define a source profile (this can be copied from the grid) and specifies a target value. A new profile is created that can be pasted back into the grid.

Next step in to allow the user to update the Mdx script for a cube thereby creating saved calculations in the model.....

SAP Business Warehouse

Tue, 27 Nov 2007 10:48:22 GMT

It has been too long since I last updated my blog, not because of a lack of desire, nor even a lack of anything to say - I have just been too busy!

A large part of that busyness has been associated with upgrading the iT-Workplace Mdx query capability to support SAP Business Warehouse. I have to admit that when I first started this project I was sceptical about BW as an Olap data source. I had heard quite a bit of negative response to the performance and architecture of BW therefore started the work with a fairly low expectation. Having worked with it for a while both in test and production environments I am now more positive.

SAP have certainly taken an unusual approach to implementing Olap for people who come from a Microsoft background. The SAP tools allow you to build queries that effectively define a sub-cube for analysis. This allows the performance of the implementation to be managed by limiting the amount of data a user will navigate at a given point in time. The downside is in the flexibility to investigate data which exists in the cube but is outside of the query definition. Given this restriction the performance of the resulting "views" is quite adequate for typical Olap navigation without being stellar.

The queries bring on their own complications when using Mdx due to some SAP specific extensions that allow you to define the "variables" that limit the scope of the query. SAP's core Mdx implementation is also very basic when compared to Analysis Services but then this is also true of TM1.

From the exposure I have had so far I would say the biggest problem with SAP BW is actually in the warehouse administrators unerstanding of what makes a usable cube/query. Most of the data I have worked could have been improved greatly if some usability testing had been done with end users. This is not of course a problem that is unique to SAP, I have seen some apalling Analysis Services implementations in my time, but because the SAP world is quite closed there does seem to be a lag in design skill for BW implementations.

I would love to implement the warehouse experiment into BW some time to see how easy it would be to create a rich Olap experience for end users. Somehow I suspect that I won;t have time to do this in the near future........

Insider view on Xmla Council

Sat, 08 Sep 2007 13:03:43 GMT

I had a telephone conversation with Amyn Rajan of Simba technologies this week. Amyn, and Simba, was the Xmla council for a large part of its existence and he was kind enough to share some insight on what happened to the organisation. His thoughts are expressed on his blog at http://blogs.simba.com/

I am not as optimistic about the state of Xmla/Mdx as Amyn. There are some fairly rudimetary differences in the way that different vendors have implemented the basic Xmla specification (for example the SAP implementation uses different restrictions to Microsoft's) and these make working with the base standard difficult. Surely it should be possible to take any Xmla "consumer" that supports the 1.1 specification and connect it to any "provider".

Further to this Amyn's notes highlight the bigger problem for interoperability which is that the provider is currently unable to enumerate its support of Mdx. A consumer should be able to query whether a provider support a particular function, or option (e.g. does the provider support the POST option in the Hierarchize function or will the provider allow 2 hierarchies from the same dimension to be displayed on the same axis). Without these interfaces provider intelligence has to be embedded into the consumer software pretty much guaranteeing maintenence problems as the providers evolve.

Without an active standards body it is likely that Xmla and Mdx will drift further and further back into the world of proprietary software.

Eric Nelson Blog

Wed, 01 Aug 2007 19:19:08 GMT

I just noticed (a bit late it has to be admitted) that Eric Nelson has given iT-Workplace a nice mention on his blog (http://blogs.msdn.com/ericnel/archive/2007/07/23/uk-isv-delivers-mdx-query-builder-control.aspx) - Thanks Eric!

More on Xmla

Fri, 27 Jul 2007 13:02:21 GMT

Yesterday I spent the day at Microsoft in Reading working with one of their SAP specialists. The upshot is that we have actually been able to connect to and retrieve data from SAP BW 3.5!

Apart from a couple of security problems the main stumbling block was the SAP implementation of "GetMembers" via the member schema rowset calls. The SAP documentation for this is quite strangely unhelpful and it appears to be the only schema rowset that they have implemented differently to Microsoft Analysis Services.

Since my last piece on Xmla Chris Webb has also blogged on the subject http://cwebbbi.spaces.live.com/blog/cns!7B84B0F2C239489A!1294.entry. I'm not sure that I agree with his conclusion that Microsoft should be the company to re-form the Xmla council. My experience of working with Xmla indicates that they were actually the only founder member of the council who properly implemented the specification and took the process seriously. I would like to see the other founder members, possibly joined by Pentaho (Mondrian) stepping up to the mark.

I do think that Microsoft own the current benchmark for interoperability in the form of their ADOMD.Net provider. A standard built around "does it work with ADOMD.Net?" would be widely acceptable from a customer perspective and would allow a significant number of mix and match application possibilities.

Reporting Sevices (BIDS) Addin

Mon, 18 Jun 2007 08:19:36 GMT

It turns out that creating an addin for Visual Studio is now quite easy! There is a nice project template to get you started and if the objective is simply to add stand-alone applications to the menu system the rest is quite straightforward. iT-Workplace now has a shiny new Query add-in available for developers who work with BIDS and Reporting Services. The tool is delivered as part of BIDS rather than being integrated into the Reporting Services UI (I will be looking at whether that is possible next).

I would like to thank Darren Gosbell for his assistance on the install procedure for this project. I would also recommend anyone who has not been there to visit the BIDS Helper project which Darren helps to co-ordinate. This project has integrated a selection of useful SSAS utilities that "Extend and enhance the functionality of the SQL Server BI Development Studio"

Reporting Services Mdx add-in

Wed, 30 May 2007 08:11:08 GMT

I have a request for help...

I was involved in a discussion the other day with Chriss Webb http://cwebbbi.spaces.live.com/ He suggested that it would be really good to create an addin for Reporting Services that provides better access to Analysis Services data. This is clearly a desireable thing to have BUT I am not quite sure where to start.

I would be very interested to hear from anyone who has written a reporting services add-in with any information on how to get started. I would also like to hear from anyone who has strong feelings about what they would like to see in such an add-in.

If it looks feasible I might well give it a go!

Query Builder Launched!

Wed, 23 May 2007 17:31:31 GMT

Today we launched the Mdx Query builder project on the unsuspecting world. Officially named "Intelligencia Olap Controls" the component includes a full Office 2007 User Interface plus our unique Spreadheet Calculation Interface that allows the end user to create calculated members and cells without being exposed to any Mdx.

If you would like to try it out it is available for download from the iT-Workplace web site http://www.it-workplace.com

I am very excited about this tool. In addition to the spreadsheet stuff it includes a number of fetures that I have wanted to see in a query component for quite a while. An example of this is the work that we have done on large dimension/query support. For a start we honor the InstanceSelection attribute displaying a different selection window for large dimensions than for smaller ones. We are also able to estimate the size of the cellset that will be returned before the query is run warning the user if the resulting cellset is likely to be excessively large.

My only slight frustration is that at present because we are using the ADOMD.NET provider we only really support Analysis Services. The tool will pop up our old UI for databases using an ODBO provider but we really need to include ODBO support in the new UI.

Query Builder Sub-Project

Wed, 07 Feb 2007 06:59:29 GMT

Mdx Query Builder Component

Doing a Google search on combinations of Mdx, Olap, Query Builder and Component there surprisingly does not appear to be much in the way of components available for consultants to use. I am therefore considering separating out the query builder from Intelligencia to make it available to others.

Would this be of interest to you?

It will probably take a couple of iterations to get the Api cleaned up and documented for general use so I am looking for some enterprising individuals who would like to get involved as early adopters. The end product would be sold on a typical component licensing model (Sorry I can't post this work for free download!) so the incentive here is that the early adopters will get free licenses and credit for the help provided.

Send an email to info@it-workplace.com if you are interested in getting involved.