Report specific variables from custom-energy-simulation

pzac · January 18, 2022, 6:31am

Hi,

I’ve been running the custom energy simulation recipe both locally and through the web. The one thing I have noticed is that the time to read the sql results locally is essentially the same as running the analysis locally using ladybug tools only. Ideally I would like to report a handful of annual results from the results database and report these only.

I have played around with modifying the recipe (using the Annual Energy Use recipe as an example of how things change) and have developed the following. Where I am stuck is pushing this to the platform. I cannot seem to generate an API key to allow me to do this.
entry.py (2.8 KB)

Hoping someone can be of assistance.

Thanks!
Pete

mostapha · January 19, 2022, 3:09pm

Hi @pzac! Great to see a modified recipe. I’ll be happy to help you push it to Pollination. Do you have a fork somewhere on GitHub? Are you setting up the CI similar to the ones for ladybug tools recipes?

chriswmackey · January 28, 2022, 11:09pm

Hi @pzac , Thanks for the feedback. Regarding this:

Why don’t you set up your Simulation Parameters to only request these handful of annual results that you are interested in? With the components in the HB-Energy 5::Simulation tab, you have full control via the simulation parameters to specify just the outputs that your want in the SQLite and the interval that you need the results at (eg. annual, monthly, daily, hourly). Setting up the simulation parameters like this can dramatically reduce the size of the output SQLite file and the time it takes to parse it, particularly if you use Annual or Monthly reporting intervals.

You’ll see that both the annual-energy-use recipe and the custom-energy-sim recipe have an input for simulation parameters. So making use of those might be a lot easier than trying to make a tweaked version of those recipes.

pzac · February 1, 2022, 9:28pm

Thanks Chris, understand what your saying here and I will give it a go. I imagine reducing the sqlite file from 8MB to less than 50kB would have big improvements in parsing.

Would there ever be a desire to report as collated .csv as per TT toolbox?

pzac · February 1, 2022, 9:29pm

Thanks for your response Mostapha, I ended up using LB tools components and running locally for now as not enough time to play around with recipe creation. Would be interested in learning more so will keep an eye on the forums for more custom recipes.

mostapha · February 2, 2022, 6:06pm

That’s something that you can do with an app. Here is an example for daylight studies which builds a Pandas DataFrame under the hood which is similar to the csv file which can also be downloaded from this button.

Here is the source code:

github.com

pollination/sample-apps/blob/87884fc2184773cf122541306543f8d1d1b883e1/design-explorer/app.py#L138-L157


      
          def post_process_annual_metrics(job_id) -> pd.DataFrame:
              data = []
              job_folder = pathlib.Path('data', job_id)
              job_folder.mkdir(parents=True, exist_ok=True)
              for run_folder in job_folder.iterdir():
                  if run_folder.is_file():
                      continue
                  res_dict = {'run-id': run_folder.stem}
                  # iterate through metrics
                  metrics_folder = run_folder.joinpath('udi')
                  for p in metrics_folder.iterdir():
                      if p.is_file() and p.suffix == '.udi':
                          grid_name = f'avg_udi_{p.stem}'
                          grid_mean_df = pd.read_csv(p, header=None).mean()[0]
                          res_dict[grid_name] = grid_mean_df
                  data.append(res_dict)
              df = pd.DataFrame(data)
              df['avg_udi_total'] = df.set_index('run-id')\
                  .select_dtypes(include=np.number).mean(axis=1).reset_index(inplace=False)[0]
              return df

I unaasign myself for now. Let me know if you have any follow up questions.