Sammy is a project manager who works for Professional Service Industries (PSI) in their Cleveland Construction Services Department.  He is responsible for supporting PSI technicians, managing and troubleshooting different projects and reviewing on-site daily field reports. On average he reviews and approves 20 to 30 reports every day that are published on Construction Hive.

Sammy says “I’ve been utilizing Construction Hive to forward all reports to the required project contacts with just two clicks. I believe Construction Hive is a good tool especially with the 2000+ reports our department distributes monthly.  Most of my clients appreciate the Construction Hive services and enjoy receiving their project daily field report within 36 hours after PSI completing the site visit.”

The 1,000,000 document milestone was achieved two and a half years after Construction Hive entered private beta. These documents published cover a broad range of material test reports and field inspection reports in relation to more than 18,000 construction projects in the United States and Australia.

Most documents published by a user in 2011

We would also like to congratulate Glenn Cebrun, Construction Services Department Manager with PSI in Houston who published 12,879 document revisions in 2011, the most of all Construction Hive users.

Glenn says “I typically review and publish reports  4-5 hours a day, 4 days a week. In the Houston area several of our large clients, Harris County, MD Anderson Cancer Center and the City of Houston require reports to be published to their project sites.  With Construction Hive I can do this relatively fast.  Our clients also appreciate the ability to set their own email notification frequency provided by Construction Hive.”

Most documents received by a user in 2011

And finally congratulations to Gordon Peterson who received 30,707 document revisions last year, more than any other user.

Gordon works for Northgate Constructors where he is Construction Quality Control Manager for the  DFW Connector project. This $1 billion design-build project involves rebuilding portions of four highways, two interchanges, and five bridges on an eight-mile stretch of roadway. At the widest point, the highway corridor will be 24 lanes wide.

Gordon was responsible for the implementation of the project quality program by managing job site quality supervisors, engineers, and technicians. Gordon developed project-specific programs, including construction quality control procedures, into an ISO 9000 certified Quality Management Plan. Gordon also helped ensure that the project was being built according to the plans and specifications, that testing is performed per applicable standards, and quality related issues were documented and resolved.

Some other interesting numbers from Construction Hive’s private beta thus far include; more than 400 report publishing users, 5,500 registered users and over 27,000 document recipients in total. All in all an impressive set of statistics for two years of operation that the whole Construction Hive team is very proud of.

The article talks about the successful  implementation of a range of Spectra QEST technologies on the high profile DFW Connector project, including Construction Hive. Read the full article here.

Then again, nested transactions in SQL Server are not as great as I remember them either, so you should be asking yourself if you need them after all.

SQL Server and nested transactions

In my mind, nested transactions in SQL worked something like this:

CREATE PROCEDURE #createFoo (@error int output)
AS
BEGIN TRAN createFoo
-- Create the foo...
SELECT @error = @@ERROR
IF @error = 0
 COMMIT TRAN createFoo
ELSE
 ROLLBACK
GO

CREATE PROCEDURE #updateFoo (@error int output)
AS
BEGIN TRAN updateFoo
-- Update the foo...
SELECT @error = @@ERROR
IF @error = 0
 COMMIT TRAN updateFoo
ELSE
 ROLLBACK
GO

BEGIN TRAN
DECLARE @err1 int, @err2 int
EXEC #createFoo @err1 output
EXEC #updateFoo @err2 output
IF @err1 = 0 AND @err2 = 0
 COMMIT
ELSE
 ROLLBACK

Here, we can call #createFoo or #updateFoo alone or, as above, wrap them into a larger outer transaction. But in reality, they’re too painful to use like that. Why? Because in the background, or at least for all intents, there’s only ever one transaction.

For example, if you rollback an inner transaction in one of the stored procs above, you’ll get the error:

Msg 266, Level 16, State 2, Procedure #updateFoo, Line 0
Transaction count after EXECUTE indicates a mismatching number of BEGIN and COMMIT statements. Previous count = 1, current count = 0.

What’s more, the outer transaction can neither COMMIT nor ROLLBACK, because after one rollback, the whole lot are gone:

Msg 3902, Level 16, State 1, Line 8
The COMMIT TRANSACTION request has no corresponding BEGIN TRANSACTION.

It bears repeating: there is only ever one transaction. COMMITs in inner transactions will basically be ignored; a ROLLBACK at any point will rollback all transactions to that point (or, as I’ve said, the one transaction), throwing any code outside of that procedure into total disarray.

So, I am no longer a fan of nested transactions. I don’t know why I ever liked them; perhaps I was using them wrong, or they never rolled back (the one case where it all goes swimmingly; but then, if you’re not prepared for a rollback, why are you doing transactions?)

NHibernate and nested transactions

NHibernate also technically “allows” nested transactions, but they are just as annoying as they are in SQL Server. Consider this code:

using (var outerTX = UnitOfWork.Current.BeginTransaction())
{
  // Create foo.
  using (var nestedTX = UnitOfWork.Current.BeginTransaction())
  {
    // Update foo.
    nestedTX.Commit();
  }
  outerTX.Commit();
}

In this case too, there is only ever one transaction. However, functionally this is even more limited than SQL Server. SQL Server maintains an internal transaction count (in @@TRANCOUNT) so that nested transactions are at least simulated to some degree, but NHibernate does not. This means that the one transaction is destroyed on both Commit() and Rollback(), meaning that the outer transaction Commit() will always fail.

While this limitation cannot really be gotten around, there are at least three solutions that may be acceptable.

Solution 1: TransactionScope()

If you want your NHibernate actions to be performed more directly within a SQL Server transaction, you can use TransactionScope. For example:

using (var transactionScope = new TransactionScope())
{
  using (var tx = UnitOfWork.Current.BeginTransaction())
  {
    ... do stuff
    tx.Commit();
  }

  using (var tx = UnitOfWork.Current.BeginTransaction())
  {
    ... do stuff
    tx.Commit();
  }

  transactionScope.Commit();
}

I’m quite unconvinced about this solution however. For one, I’m not sure if TransactionScopes can themselves be nested, or what effect this has. And does a Rollback() just cause the same issues as it does in normal SQL Server statements? I can’t recall. This solution allows us to nest NHibernate transactions within a SQL Server transaction, but to what end?

In addition, this is basically SQL Server specific, which limits the wonderful range of DBMS that NHibernate can connect to. You also have to enable MSDTC to use TransactionScope, which is a hassle in itself, and even on a local server it seems to too easily want to promote itself to a distributed transaction, which has performance costs, and so on and so on… I did not want to get involved.

Solution 2: With.Transaction()

If you’re using Rhino Commons this is automatically available, though it would be quite easy to make your own version. A usage example is shown below:

With.Transaction(delegate
{
  // Create foo.
  With.Transaction(delegate
  {
    // Update foo. Throw an exception to rollback this transaction and the outer one.
  });
});

The original source is here, but basically it works with the one transaction limitation by:

• doing nothing if you’re already in a transaction, or
• creating a new transaction if you’re not. Also,
  • if a new transaction is created, any exception will cause the transaction to be rolled back.
  • if an exception is not thrown, the transaction is committed after the delegate has completed.

So although we have no direct control over the commit/rollback over the transaction, it generally does exactly what we want it to do. It allows us to write functions which effectively wrap their NHibernate calls in transactions, but we don’t need to explicitly worry about whether we’re part of a larger transaction at that point or not.

Generally, I still like to use explicit transactions in the outer transaction, as that’s where we’ll want to control whether to commit or rollback the whole lot.

Solution 3: Create a new session

A new NHibernate session isn’t really nested transactions at all, but it does allow for multiple transactions. That’s because the one transaction limit is actually one transaction per session. Therefore, creating a new session will allow you to always create a new transaction without fear that the Rollback() will harm any other transactions you’ve created.

In this case, of course, the new transaction is in no way a part of any “outer” transaction you’re running, so it is committed or rolled back entirely independently. Depending on how your code is structured, this solution may be sufficient if all you’re looking for is a way to couple together nested functions which each need to create transactions.

Again, using Rhino Commons this is most easily achieved with a new UnitOfWork:

using (UnitOfWork.Start())
{
  // Create foo.
  using (UnitOfWork.Start(UnitOfWorkNestingOptions.CreateNewOrNestUnitOfWork))
  {
    // Update foo. Note that even if you rollback here, the outer transaction will still commit
    // and the foo will be created.
  }
}

Hope that was helpful!

- Gavin

Analytics for document publishers

We have now introduced Analytics for document publishers to track their activity on Construction Hive. This includes metrics on the reports published for a given period, by location and report type, etc.

This new capability is available for publishing company administrators and those that have been assigned the new analytics role. We have opted for the use of the Google Visualization API for the tabulated and graphical display of this information. While the current capability is limited to providing analytics about the utilisation of the Construction Hive service the underlying framework that has been put in place forms the foundation of a more comprehensive analytics piece for the monitoring of construction quality related metrics such as historical constructional material (concrete, aggregate, soils, asphalt, etc.) performance. You can expect to see this become available in a future release.

Bulk download of documents

Document consumers have been telling us that while they like the ability to search and view documents online they sometimes also have a need to store these documents in a local file system or other document management system. While Construction Hive’s integration service can provide an integration point with third party systems, this is not always practical or the configuration effort justified. To handle these circumstances we have provided a bulk download feature for documents. This is available from the Search Results page.

Stability & Robustness

As traffic on Construction Hive has increased we have identified some areas where the site’s performance could be improved. At peak load times some users have been reporting slow/unresponsive behaviour. We have been working hard to address this issue and have included some modifications in this build which will help improve this . Over the coming weeks we will be monitoring the site performance and making further refinements.

This is the first major implementation of the Construction Hive integration services. It has some challenging requirements involving two target systems – one for PDF versions of reports and another for detailed materials testing results.

For more information please see the press release.

One of the primary goals we outlined when embarking on the design of Construction Hive was to provide a platform that could leverage the data that was the source of many documents and reports distributed in the construction industry. This Construction Hive release brings us a step closer to realising that goal.

Working with Metadata

Currently some Beta users are publishing test reports from QESTLab directly to Construction Hive. These reports are based on data collected in the sampling and testing of construction materials. This data identifies the material being tested, the material supplier and source, where on a job site the material has been placed and the materials physical properties. This information is collectively referred to as metadata and it is now being published and stored with the reports on Construction Hive.

Show me the data

When you now view a document on Construction Hive you can see the most important metadata relating to that report right on the preview screen, without the need to actually open or download the report. The example below shows a concrete test report with details of the supplier, mix, location, truck, test results, etc. displayed along side the preview image.

Searching using Metadata

The association of the metadata with the document allows for the ability to search for a document based on the metadata itself. Up until this release the Construction Hive searching could already be performed on some basic metadata including the date a document was published, the project name, tags and status. With the new release this has been extended to search by a much larger metadata set including specialised data that relates to the specific type of document.

For example, in the image above you can see specialised searching is available to retrieve documents that relate to a specific supplier of a construction material (Agg/Soil, Concrete or Asphalt), the material code or name, location, lot number, etc.

Integration Framework

Another major capability introduced in this release is the Construction Hive Integration Service. This service allows for the integration with third party systems, where documents and metadata are pushed from Construction Hive using web service calls or FTP.

This service will be used for integrating with third party document management systems being used on major construction projects, GIS systems, and the like. The first such integration is already taking place for a major road project in Texas. I will post more on that in the near future.

And the other stuff

Amazon S3

Our plans call for Construction Hive to be highly scalable as we expect to be managing millions of documents in the future. Part of the technological mix required to achieve this is the ability to efficiently and securely manage the storage and retrieval of large amounts of binary objects (e.g. documents in PDF and other formats). Traditional databases do not perform this task particularly well so we have elected to utilise Amazon S3 for this. What, you may ask, has the world’s largest online retailer got to offer in this area? Well, as it turns out, a great deal: they make available the same technologies they use to run their enormous cloud based business to other developers.

We implemented the use of Amazon S3, in this release. What does all this mean to the end user experience? Apart from the reliability and scalability you expect, very little as the use of the technology is transparent to you.

User Interface Improvements

This release also include a range of user interface improvements, just making stuff feel a little more slick. One noticeable change is the introduction of a hover box for document links, showing some basic details about the document.

We hope our users enjoy these new features, and as always welcome your comments.

The press release for the beta launch can be found here.