Background

Today’s post was inspired by a problem I encountered on the first day of the BI Immersion class. To make things easier for folks to have a working environment in which to try out the topics we discuss during the BI Immersion class, I recommend they get a freely available VHD with the entire Microsoft BI stack installed and configured. It’s evaluation software so it only works for a limited time, but it should be enough time for learning until you get your own permanent environment set up. Also, it’s not quite the entire BI stack, but we fix that during class by adding PowerPivot for SharePoint and adding another Reporting Services instance in SharePoint integrated mode. (Side note: Adding PowerPivot for SharePoint to an existing farm can be challenging – may need to do some fiddling to get it working.)

The Problem

As I was demonstrating Reporting Services (native mode) using the aforementioned VHD, I noticed that the Data Feed button was missing from the HTML Viewer toolbar. Yet, it was there in the toolbar when using Reporting Services in SharePoint integrated mode. You use this button to export data to PowerPivot for Excel – or any application that can consume an ATOM data feed. Where did the button go?

The Solution

My guess is that this VHD was not created from scratch using SQL Server 2008 R2, but rather migrated from an earlier version and somehow the RSReportServer.config file didn’t update properly. When I looked at the file for clues about the data feed button, I noticed that a rendering extension was missing. I added it back in as shown below (see the highlighted text), then opened up Report Manager to look at a report. Hooray – the button is back!

(Sorry about the lack of wrapping – I’ll fix it later. Have to get to class!)

<Render>
<Extension Name="XML" Type="Microsoft.ReportingServices.Rendering.DataRenderer.XmlDataReport,Microsoft.ReportingServices.DataRendering"/>
<Extension Name="NULL" Type="Microsoft.ReportingServices.Rendering.NullRenderer.NullReport,Microsoft.ReportingServices.NullRendering" Visible="False"/>
<Extension Name="CSV" Type="Microsoft.ReportingServices.Rendering.DataRenderer.CsvReport,Microsoft.ReportingServices.DataRendering"/>
<Extension Name="PDF" Type="Microsoft.ReportingServices.Rendering.ImageRenderer.PDFRenderer,Microsoft.ReportingServices.ImageRendering"/>
<Extension Name="RGDI" Type="Microsoft.ReportingServices.Rendering.ImageRenderer.RGDIRenderer,Microsoft.ReportingServices.ImageRendering" Visible="False"/>
<Extension Name="HTML4.0" Type="Microsoft.ReportingServices.Rendering.HtmlRenderer.Html40RenderingExtension,Microsoft.ReportingServices.HtmlRendering" Visible="False"/>
<Extension Name="MHTML" Type="Microsoft.ReportingServices.Rendering.HtmlRenderer.MHtmlRenderingExtension,Microsoft.ReportingServices.HtmlRendering"/>
<Extension Name="EXCEL" Type="Microsoft.ReportingServices.Rendering.ExcelRenderer.ExcelRenderer,Microsoft.ReportingServices.ExcelRendering"/>
<Extension Name="RPL" Type="Microsoft.ReportingServices.Rendering.RPLRendering.RPLRenderer,Microsoft.ReportingServices.RPLRendering" Visible="False" LogAllExecutionRequests="False"/>
<Extension Name="IMAGE" Type="Microsoft.ReportingServices.Rendering.ImageRenderer.ImageRenderer,Microsoft.ReportingServices.ImageRendering"/>
<Extension Name="WORD" Type="Microsoft.ReportingServices.Rendering.WordRenderer.WordDocumentRenderer,Microsoft.ReportingServices.WordRendering"/>
<Extension Name="ATOM" Type="Microsoft.ReportingServices.Rendering.DataRenderer.AtomDataReport,Microsoft.ReportingServices.DataRendering" Visible="false"/>
</Render>

Downloads available:

• Presentation slides
• Demonstration project

Obtaining State Shapefiles

To start, the process is similar to the one I described in my previous post on working with spatial data. This time my goal is to create a map of two states – Nevada and California. The best place to download free shapefiles for US states is the U.S. Census Bureau where I used the link to download by state. I found the Nevada directory and then used the topmost directory 32 where there many files to download. I chose tl_2010_32_state00.zip. Similarly in the California directory, I used the topmost directory to find and download tl_2010_06_state00.zip.  Then I extracted each zip file to its own directory on my computer. They each contain a variety of files, including SHP and DBF files that I could use for a map using the ESRI shapefile data source option if I want to create a separate map for each state. But I want to combine these maps, so my mission is not yet complete.

Converting Shapefiles to SQL Spatial Data

My next step is to use a tool called Shape2SQL to extract the spatial data from the shapefile into a SQL Server table. Pinal Dave (blog|twitter) has a tutorial explaining how to do that. I had the same experience that I described in my last post – I had to clear the Create Spatial Index checkbox to get the table to load properly for the first state. When I loaded the second state, I had to clear the Replace Existing Table checkbox. Now I have both states in a table called State, as shown below, with a geom column having the SQL Geometry data type.

Creating the Multi-State Map

Now that I have my spatial data in a table, I’m ready to create the map. In Business Intelligence Development Studio, I added the map to my report which launched the Map Wizard and then I used the following steps:

1. On the Choose a source of spatial data page of the wizard, select SQL Server spatial query, and click Next.
2. On the Choose a dataset with SQL Server spatial data page, select Add a new dataset with SQL Server spatial data.
3. On the Choose a connection to a SQL Server spatial data source page, select New.
4. In the Data Source Properties dialog box, on the General page, add a connecton string like this (changing your server name if necessary):
   Data Source=(local);Initial Catalog=SpatialData
5. Click OK and then click Next.
6. On the Design a query page, add a query for the country shape, like this:
   SELECT  NAME00, geom FROM State
7. Click Next. The map wizard reads the spatial data and renders it for you on the Choose spatial data and map view options page, as shown below.

Of course, you can use these technique with any shapefiles that you can locate. It’s not limited to the United States. For example, if you want to create a map of multiple European countries or a combination of states in India, you could use a similar process by downloading shapefiles from either the Global Administrative Areas spatial database or the Natural Earth database.

When designing a report, you have to be really careful with the layout of items in the body. As you move items around, the body will expand to accommodate the space you’re using and you might eventually tighten everything back up again, but the body doesn’t automatically collapse. One of my favorite things to do in Reporting Services 2005 – which I dubbed the “vacu-pack” method – was to just erase the size property of the Body and let it auto-calculate the new size, squeezing out all the extra space. Alas, that method no longer works beginning with Reporting Services 2008. Even when you make sure the body size is as small as possible (with no unnecessary extra space along the top, bottom, left, or right side of the body), it’s important to calculate the body size plus header plus footer plus the margins and ensure that the calculated height and width do not exceed the report’s height and width (shown as the page in the illustration above). This won’t matter if users always render reports online, but they’ll get extra pages in a PDF document if the report’s height and width are smaller than the calculate space.

Beginning the Investigation

In the situation that I was troubleshooting, I checked the properties:

So I calculated the total width using Body Width + Left Margin + Right Margin and came up with a value of 10.7 inches. And then I calculated the total height using Body Height + Page Header Height + Page Footer Height + Top Margin + Bottom Margin and got 7.6 inches. Well, page sizing couldn’t be the reason for the extra page in my report because 10.7 inches is smaller than the report’s width of 11 inches and 7.6 inches is smaller than the report’s height of 8.5 inches. I had to look elsewhere to find the culprit.

Conducting the Third Degree

My next thought was to focus on the rendering size of the items in the report. I’ve adapted my problem to use the Adventure Works database. At the top of the report are two charts, and then below each chart is a rectangle that contains a table.

In the real-life scenario, there were some graphics present as a background for the tables which fit within the rectangles that were about 3 inches high so the visual space of the rectangles matched the visual space of the charts – also about 3 inches high. But there was also a huge amount of white space at the bottom of the page, and as I mentioned at the beginning of this post, a second page which was blank except for the footer that appeared at the bottom. Placing a textbox beneath the rectangles to see if they would appear on the first page resulted the textbox’s appearance on the second page. For some reason, the rectangles wanted a buffer zone beneath them. What’s going on?

Taking the Suspect into Custody

My next step was to see what was really going on with the rectangle. The graphic appeared to be correctly sized, but the behavior in the report indicated the rectangle was growing. So I added a border to the rectangle to see what it was doing.

When I added borders, I could see that the size of each rectangle was growing to accommodate the table it contains. The rectangle on the right is slightly larger than the one on the left because the table on the right contains an extra row. The rectangle is trying to preserve the whitespace that appears in the layout, as shown below.

Closing the Case

Now that I knew what the problem was, what could I do about it? Because of the graphic in the rectangle (not shown), I couldn’t eliminate the use of the rectangles and just show the tables. But fortunately, there is a report property that comes to the rescue: ConsumeContainerWhitespace (accessible only in the Properties window). I set the value of this property to True. Problem solved. Now the rectangles remain fixed at the configured size and don’t grow vertically to preserve the whitespace. Case closed.

Let’s say that I have a simple table like this:

To enable interactive sorting, I open the Text Box properties for each of the column headers – the ones in the top row. Here’s an example of how I set up basic interactive sorting:

Now when I preview the report, I see icons appear in each text box on the header row to indicate that interactive sorting is enabled. The initial sort order that displays when you preview the report depends on how you design the report. In this case, the report sorts by Sales Territory Group first, and then by Calendar Year.

Interactive sorting overrides the report design. So let’s say that I want to sort first by Calendar Year, and then by Sales Territory Group. To do this, I click the arrow to the right of Calendar Year, and then, while pressing the Shift key, I click the arrow to the right of Sales Territory Group twice (once for ascending order and then a second time for descending order). Now my report looks like this:

This technique only seems to work when you have a minimum of three columns configured with interactive sorting. If I remove the property from one of the columns in the above example, and try to use the interactive sorting on the remaining two columns, I can sort only the first column. The sort on the second column gets ignored. I don’t know if that’s by design or a bug, but I do know that’s what I’m experiencing when I try it out!

I agree with Doug Lane (blog | twitter), who pointed out in his T-SQL Tuesday post that it’s best if you use the database engine to calculate aggregates rather than use aggregate functions in Reporting Services. There are some things the database engine can simply do better, and we should use the best tool for the job. But an even better way to work with aggregated data, in my opinion, is to put data into a cube and let Analysis Services do the calculations. Either way, whether you use a relational source like SQL Server or a multidimensional source like Analysis Services, the point is that the back-end engines will always outperform Reporting Services. On the back-end, Analysis Services should outperform SQL Server if the cube and the report query are both designed properly.

That said, there is some trickiness involved when  retrieving and using aggregations in reports. I was having a déjà vu moment as I was writing this blog and sure enough I found that I had touched on the subject several years ago in my blog post, Showing Server Aggregations in Reporting Services. The post was written in reference to Reporting Services 2005, but the principles still apply. In today’s post, I’ll revisit the same query but add in some pictures this time to help clarify.

The Problem

If I create a table using the query from the referenced post, and if I add a total row using the Reporting Services SUM function, I get this result:

The SUM function would work just fine if I were to use an additive measure, like SalesAmount, but the SUM function doesn’t work at all with non-additive measures like Gross Profit Margin. It adds up the percentage values to produce the total row value but it really needs to compute the sum of sales and the sum of costs separately and then do the math to compute the correct result.

The Solution

Instead of using the SUM function, we can use the AGGREGATE function instead. That is we replace this expression:

=Sum(Fields!Gross_Profit_Margin.Value)

with this expression:

=Aggregate(Fields!Gross_Profit_Margin.Value)

Where’s the Value?

When I make the change to the expression and preview the report, I find that I have a new problem – the total row is now missing a value completely!

The reason is that the Aggregate function requires me to use a query in design mode only, not in query mode. (At least that’s the case in Reporting Services 2008 R2. I don’t remember that this was true in Reporting Services 2005, but I don’t have an instance for that version handy at the moment to try it out. I think, from reading my prior post, that it worked in either mode, but I can’t prove it at the moment!) In other words, I can’t write the MDX manually in query mode – I must use the drag-and-drop interface to create my query. That’s fine for simple queries, but it won’t work when you need to add in calculations or otherwise make adjustments to the auto-generated query.

The Complete Solution

So once I switch the query to design mode like this:

And then preview the report, I now get the desired result.

As I mentioned above, this will only work when the query is in design more. And there’s another requirement–the tablix must have at least one group defined. The benefit, though, is a faster report than you’ll get from running a comparable relational query. You won’t notice the difference in the AdventureWorks databases, but in that case, Reporting Services will perform aggregate functions reasonably well anyway. But if your data source has high volumes of data, then using a combination of cubes and Reporting Services will deliver much faster reports.

As with many problems, Reporting Services gives us more than one way to apply OR logic in a filter. If I want a filter to include this value OR that value for the same field, one approach is to set up the filter is to use the IN operator as I explained in Part 1 of this series. But what if I want to base the filter on two different fields? I  need a different solution.

Using the AdventureWorksDW2008R2 database, I have a report that lists product sales:

Let’s say that I want to filter this report to show only products that are Bikes (a category) OR products for which sales were greater than $1,000 in a year.

If I set up the filter like this:

Then AND logic is used which means that both conditions must be true. That’s not the result I want.

Instead, I need to set up the filter like this:

The OR logic needs to be part of the expression so that it can return a Boolean value that we test against the Value. Notice that I have used =True rather than True for the value. The filtered report appears below. Any non-bike product appears only if the total sales exceed $1,000, whereas Bikes appear regardless of sales. (You can’t see it in this screenshot, but Mountain-400-W Silver, 38 has sales of $923 in 2007 but gets included because it is in the Bikes category.)

As part of my standard operating procedure when demonstrating report snapshots, I point out this label, so I was rather taken aback when I didn’t see it in the demonstration I was preparing. It sort of upset my routine, and I’m rather partial to my routines. I thought perhaps I wasn’t looking in the right place and changed Report Manager from Tile View to Detail View, but no – that label was still missing. In the grand scheme of life, it’s not an earth-shattering change, but you’ll have to look at the Modified Date in Details View to know when the snapshot was run. Or hope that the report developer included a textbox to show the execution time in the report. (Hint: this is a good time to add this to your list of report development best practices, whether a report gets set up as a report snapshot or not!)

A snapshot from the past

In case you don’t remember how a snapshot appeared in Report Manager back in the old days (of SQL Server 2008 and earlier), here’s an image I snagged from my Reporting Services 2008 Step by Step manuscript:

A snapshot in the present

A report server running in SharePoint integrated mode had no such label. There you had to rely on the Report Modified date-time stamp to know the snapshot execution time. So I guess all platforms are now consistent.

Here’s a screenshot of Report Manager in the 2008 R2 version. One of these is a snapshot and the rest execute on demand. Can you tell which is the snapshot?

Consider descriptions as an alternative

So my report snapshot demonstration has one less step, and I’ll need to edit the Denali version of the Step by Step book. Things are simpler this way, but I sure wish we had an easier way to identify the execution methods of the reports. Consider using the description field to alert users that the report is a snapshot. It might save you a few questions about why the data isn’t up-to-date if the users know that something changed in the source of the report. Notice that the full description doesn’t display in Tile View, so keep it short and sweet or instruct users to open Details View to see the entire description.

Preparing the Report

Let’s start with a simple scenario using the AdventureWorksDW2008R2 database. I set up a query that returns reseller sales by category. In my report, I have two tables: one for Bikes and Components, and the other for everything else. I want to create a filter for the first table using the IN operator, but I’ll have to create an expression for the second table to achieve the NOT IN effect.

To hold the list of values for my IN filter, I create a hidden report parameter with multi-values, without available values, and with a list of default values that includes Bikes and Components, like this:

Of course, I don’t need to manually populate the default values. I could use a query instead, but I think you get the idea.

Using the IN Operator

This step is not really required to set up the NOT IN filter, but I have included it as another example of working with the IN operator to supplement my first post in this series. In my report, I want the first table to include the values that are defined in the parameter, so I add a filter to the tablix like this:

You can also use [@FilterList] as the value if you want to type in a simple expression into the Value field. If you use the Expression Editor to set up the Value expression, and double-click on FilterList in the Parameters category, be sure to remove the (0) from the end of the expression so that the IN operator is comparing Category to all values in the parameter array, not just the first value.

Creating a NOT IN Filter

For the second table, I also add a filter, but I set it up differently. There is no such thing as a NOT IN operator, so I need to get creative. I need to come up with an expression that evaluates as True or False, and then set that up as my Value in the filter definition. Then I’ll set the Expression of the filter to True and use an = operator. The filter will keep rows where Value is True and exclude rows which cause Value to be False.

For Expression, note that I have =True and not just True. This is important because True by itself will be interpreted as a string instead of a Boolean data type.

I use the InStr function in the value to compare the current row’s Category to the FilterList which I convert from an array to a comma-delimited string by using the Join function. The InStr function returns 0 if the category is not found, which is the equivalent of NOT IN. Thus, I have the expression return a True – which keeps the row in the second table.

Checking the Results

Here’s the final report:

If you’d like to have a look at how I set this up, you can download the RDL.

Have you found another way to solve NOT IN? Let me know!

Today, I happened to be working on an example of using the TOP N operator and was not successful on my first try because the behavior is just a bit different than we find when using an “equals” comparison as I described in my first post in this series. In my example, I wanted to display a list of the top 5 resellers in the United States for AdventureWorks, but I wanted it based on a filter. I started with a hard-coded filter like this:

And received the following error:

A filter value in the filter for tablix ‘Tablix1′ specifies a data type that is not supported by the ‘TopN’ operator. Verify that the data type for each filter value is Integer.

Well, that puzzled me. Did I really have to convert ResellerSalesAmount to an integer to use the Top N operator?

Just for kicks, I switched to the Top % operator like this:

This time, I got exactly the results I expected – I had a total of 10 records in my dataset results, so 50% of that should yield 5 rows in my tablix.

So thinking about the problem with Top N some  more, I switched the Value to an expression, like this:

And it worked!

So the value for Top N or Top % must reflect a number to plug into the calculation, such as Top 5 or Top 50%, and the expression is the basis for determining what’s in that group. In other words, Reporting Services will sort the rows by the expression – ResellerSalesAmount in this case – in descending order, and then filter out everything except the topmost rows based on the operator you specify.

The curious thing is that, if you’re going to hard-code the value, you must enter the value for Top N with an equal sign in front of the integer, but you can omit the equal sign when entering a hard-coded value for Top %. This experience is why working with Reporting Services filters is not always intuitive!

When you use a report parameter to set the value, you won’t have this problem. Just be sure that the data type of the report parameter is set to Integer. Jessica Moss has an example of using a Top N filter in a tablix which you can view here.

Working with Bottom N and Bottom % works similarly. You just provide a number for N or for the percentage and Reporting Services works from the bottom up to determine which rows are kept and which are excluded.