Post process workflow

Hi @mostapha, @chriswmackey,

We are having good progress with setting up our workflow for parametric runs. Cloud simulations are saving a huge amount of time and headaches. Thanks for your effort in developing these tools and helping us at each step. The next piece of the puzzle is post-processing. Right now downloading the SQL files and converting them to excel and then processing them for visualizations are quite time and effort-intensive. Is there any way to process the sql files on the cloud itself?

@ian.vanduivenbode @jamesthomson

Hi @anubis - great to hear the overall process is working. That’s exactly why we started the development of the Pollination apps. The combination of apps and recipes gives you two options depending on the amount of work you want to do and how much flexibility do you want to have.

The first option is to modify the recipe and create a new one with the post-processing steps that you have in mind. This option has been fully supported by Pollination for a long time now and is scalable since the post-processing step will run as part of each execution and in parallel to the rest of the simulation. The challenge has been that modifying a recipe is not necessarily an easy task.

The second option which should be easier to do is to write an app: About the Apps category - It only needs some Python scripting and there are several examples already available here:

This one has a step or two for post-processing the SQL files for results visualization. Here is the source code.

The apps are not fully supported yet on Pollination but I can write you an app that does what you need and deploy it similar to the other sample apps until we officially support them. That should get you going for now.

What type of post-processing are you looking for? We also have an example for comparative studies for daylighting recipes that can be adjusted to work for energy simulation. Here is the live version and here is the source code.

:memo: At this point, the sample apps timeout is being left inactive for some time to make sure we don’t pay for something that’s not being used. One needs to refresh the page manually to have them work again. This behavior will change after we support them officially on Pollination.


Thank you so much Mostapha. We will give it a go

Sounds good! Let me know how it goes.