TIBCO Spotfire® allows us to connect to different data sources, but data from Web services are retreived in formats that can not be read by available connectors. At the same time we have encountered the need for systematic storage of certain data in its own database so this data can later analyzed in Spotfire®.
Challenge
The challenge was how the data from these systems could be analyzed with the TIBCO Spotfire® solutions. We have two data sources: Web services provider of energy source consumption API for weather forcasts and history – web portal forecast.io. In our case the quantities depend heavily on the temperature and weather in general, so weather forecast can predict the future energy consumption very well.
We used the following solutions
Data integration was done with:
- MuleSoft solutions for data integration of data from web services and weather forecast API
- TIBCO Spotfire® solutions for data analyzing and data vizualisation.
Example
We compared the quantity with temperature and other meteorological parameters and observed how the weather affects the actual energy consumption.
Weather data is fetched from portal www.forecast.io (https://darksky.net/) and systematically stored in MSSQL database. This data is then used for analysis in TIBCO Spotfire® solutions.
Let see an example:
Columns show the energy consumed, lines the temperature. Therefore we can mark certaing days if we want to examine days in detail.
For selected dates icons tell us what the weather was that day or the forcast if we mark days in the future.
Since the temperature is recorded on hourly basis each day can be analysed in detail regarding its impact on the energy consumed.
In addition to temperature we have further information about the weather, such as rainfall intensity, wind speed, and similar. These are all factors that may affect the volumes consumed. The data can be analysed retrospectively using statistical model that can predict the quantites in the future. Based on historical data it is necessary to provide other relevant input parameters that have a significant impact on the quantity, for example time profile of consumption, calendar days, seasonal factors, internal forecasts, and one time events.
It is crucial that all the information is available as soon as it is created, without additional effort, copying and manual transformation. Statistical models can be integrated in a way that their application is repeatable at any time and the results of these models can be reached in the same way as all other analyzes by users with no or less statistical knowledge.
Who should think about such solutions?
Such solutions are particularly useful for energy companies that track energy consumption at different points, but the data are accessible via web services. They dealing with short-term forecasting where errors can cause large costs and forecasting is carried out on a daily level. It is therefore essential that we can provide automation of these tasks and optimize the statistical models.
