@mostapha I tried uploading a zip file with the IDFs to pollination, it was significantly faster. Is there a way to unzip a file that has been uploaded to the cloud?
Hi, @cameron-lawrence - I created a new topic for this to be discussed separately.
The short answer is yes. You can use the Zip file as input to a recipe and unzip the files as part of the recipe, and then loop over the input files to run the simulation for each input file.
I saw this comment in your email:
I ran a quick test with 100 IDFs in a folder vs 1 zipped file containing the same set of files. 3 vs 117.2 seconds.
Depending on the overall time of the run the 2 minutes here may or may not justify writing a new recipe. One item to keep in mind is that if you run all the IDFs as part of the same run, one problematic IDF file will mark the whole run as a failed run. It will also stop the post-processing step. There are workarounds to make this work but generally, I would suggest running every model as a separate run using an optimized recipe.
@mostapha Thanks for the response. I think it makes sense to run the models individually, but upload as a ZIP file. How would we go about updating the recipe?
Hi, @cameron-lawrence,
If you run each model individually then it probably doesn’t makes sense to use the ZIP file approach because the time that you save during the process of uploading the file will be offset for the overhead by unzipping the file and picking the right file in the recipe.
Let’s start with post-processing. If you can wrap your post-processing steps in a single Python function we can take it from there. We can continue that conversation in a private message or via email.
Cheers!