The new year is bringing big changes for me. Even though I’m still in the same company we have decided to bring a totally new BI platform, the good old SAP BW plus his companion the BO Suite. As I’m not a technical expert in this platform, I’ve been blessed to be nominated the BI Lead, not only for it but also for the previous Microsoft platform. My tasks will span Project Management, Technical Lead, Developer and Support. Luckly we have contracted a SAP BI specialist that will focus on the development and support of the SAP BI platform and I will concentrate on the Project Management and Technical Lead for the SAP BI while doing little developments and support in the Microsoft BI until we can fully migrate all the developments to the SAP one. Not bad!
I know that some people will not be happy to have almost 15 years of experience in one platform, in my case Microsoft, and then all of a sudden being thrown to a different one, and as different as SAP. But I’m happy about it. For a long time I wanted to put some distance between me and the pure and hard technical side of IT and focus more on the management and leadership of technical solutions. So I’m embracing this opportunity as a challenge and as a new era in my IT career.
From now on you will see a different kind of posts, which I hope they can be useful to anyone out there, in the vast realms of the Internet. The posts will revolve around architectures, PMBOK project management comments, business gatherings and experiences with business users, etc. Expect me to share my templates and diagrams multiple times as I keep learning and evolving in this new life.
Being this the very first post of this new era, we shall start by the vision, or to be a bit more precise the system architecture.
As you will see my skills in the Microsoft BI platform will not be totally wasted. As we rely on SharePoint for our intranet, the easiest way to bring data from a SharePoint list is to use the SSIS connector and dump the data into a SQL Server table. Also I have learned that if you need to integrate with strange systems, like IP21, whose experts are outside the IT department, it’s quite easy to ask them to dump whatever data we need for our BI analysis into a table in SQL Server, and from there you can import your data into your BI system using an ETL tool. For these reasons, we will be using SQL Server as a “data middleware” or some people would like to call it a data staging area.
I do like the idea of having a place where the data stops before entering the BI platform. A place where all data has the same format, can be logged and inspected easily. This approach helps you to industrialize your architecture. The path is clear, the methods and tools are robust, and on top of all it gives you a great flexibility and peace of mind regarding version and inter-application compatibilities.
You can see SAP PI also in the architecture and some people could argue that if you have SAP PI you don’t need SQL Server as a staging/middleware. But this for me is not true. Both products could be used for the same purposes, but having the capability doesn’t mean that you should, or better abuse, it. SAP PI, as other middlewares like BizTalk, are good dealing with lots of light messages. A light message is a message carrying a small set of data with it. I always give the same example, think about an ATM network for a bank. All the transactions yield from the ATMs go to a middleware and this in return sends the messages to the appropriate back-office systems in the bank. Those transactions are high in number but low in weight. A middleware is an ideal solution for this scenario. But if you would like to kill a middleware, start making medium data transfers through it and you will see how the performance is heavily impacted. If you would like to transfer medium to high volume of data between two or more systems, there is nothing better than a good old database with an ETL tool. We could say that if the number of messages is low but the data being transfered is high, then nothing can beat a database. For these reasons I have placed SQL Server as the main “data middleware” and SAP PI on the side for those cases where is suitable to go directly for it.
That’s enough for the first post of the year 2014. I wish you a very happy new year!