No artifacts found at HBJSON file

Hi @tyler . I have just submitted a custom energy file to pollination and it says "No artifacts found at projectname.HBJSON. What are your suggestions for fixing this error? It has popped up before and a suggestion was that my constructions for the zones may be wrong…

Which project was this for? And how were you uploading this file?

It is for one in the Aurecon Group company group. Do you have access? I am sending it from the GH Plugin. It has 256 runs that are being run in a similar fashion to my previous jobs (colibri+pollination), but it is an energy model.

@tyler any update on this issue?

Hi @compdesignernz :wave:,

I looked into your issue and think I found the project and job where you encountered this error. I am not going to drop any links here as I assume you would like to keep the name of the project private.

That being said, I think the error message explains what went wrong: there is not file called FAS-BuildingMinisterial_glz0_uV0.625_gPerfLow_rPerfLow.hbjson in the Ministerial/Energy-Simulations folder for this project.

Possible Fixes

It is worth checking the script used to upload the files/start the simulation. Were you using the Pollination Grasshopper components? If you go back over your script can you check that the name was assigned properly? Did you stop the script and resume it some time afterwards which might have interrupted the upload of a specific file?

If you are using the Grasshopper plugin: @mingbo is it possible that the plugin would somehow not upload a file or submit a job before the file is uploaded?

Reproducing this bug on your end

You can view the files in your project’s folder by clicking on the Files tab right below the name of your project and then search for the folders/files you are after:

There are tonnes of files with very similar names but using CTRL + F for FAS-BuildingMinisterial_glz0_uV0.625_gPerfLow_rPerfLow.hbjson shows that no file with that name exists.

1 Like

Sorry for the delay! I realized I didn’t have access to the org and was going to try to investigate through other means but had some fixes to merge first. Thanks @antoinedao for picking it up though!

1 Like

I believe that I got @Max to add you to the org…

Oh, so you did, I wasn’t aware. Looks like access was granted on the 19th in my afternoon. That must have been after the last time I tried to access an Aurecon project. Thanks for letting me know!

Thanks @antoinedao ! I’ll have a look through again. My process for creating the files was using the Colibri plugin and using the data recorder to record the runs. I am re-running the outputs now and will see if it uploads. There are 256 parameter combinations, so there should be 256 runs, if I am not mistaken?

I’ll let you and @tyler know once I have successfully uploaded them and will see if it works.

1 Like

@antoinedao @tyler so I ran the process again, and there is definitely an issue with this. I ran the single iteration that has the parameters above (0,0.625,low,low), and that loaded in fine, but when I load the full run set, it says that it is missing that iteration (as per the above error message). I am unsure whether my file is appending additional files that don’t reference anything, or whether it is an issue on your end…

It is worth noting that I am using the same parameters, process and script for running an annual daylight sim and that worked fine, and so I am unsure as to why this error is occurring with the custom energy sim :confused: The only thing being changed is the recipe, everything else is constant.

Hi @compdesignernz, could you check a folder that you can access from following path, to see if it contains all files? GH plugin basically uploads all files in this folder.
%temp%\Pollination\prepareArtifacts

Something like this, just find a subfolder that matches the date and time when you execute the job.

1 Like

I checked this quickly on the server side last night and can confirm the file that is referenced in the Job’s failure message does exist in our blob storage.

Our best guess right now is that this is a race condition where the “create job” request hits the server before all of the artifacts have finished uploading.

It’s on my list to look into today, though.

How many .hbjson files do you expect to be under the energy-simulations directory? I count 204 files for a total of 307.46MiB in cloud storage.

1 Like

Hey, @tyler :wave:

That’s good news, I guess :sweat_smile: Glad you found the bug! I expect there to be 258 runs for the full simulation. Keep me posted on whether you get that sorted! Is there anything I can try my end to fix the error?

Well, it’s just a guess right now and it’s tough to reproduce. Could you check the directory that @mingbo suggested?

I have checked it. The /Pollination/ dir doesn’t exist…

Dang. I’m trying to replicate this now to be sure it’s the right issue. I’ll share something when I know more.

Great! Thanks for that :smiley:

I’m still not sure what the root cause is, but there definitely was a large delay between the upload and the job creation.

Our event store shows that the file was done uploading at 02:19:43 while the job was started 2 minutes earlier at 02:17:08. All of the files took over 5 minutes to upload.

Would you be able to try with smaller numbers of combinations to help us figure out when this happens? E.g. with 1/2 as many and 1/4 as many combinations?

Did you happen to notice any of the GH components having errors while submitting the job?

2 Likes

Oh wow!

Yeah I can try smaller numbers of combinations.

It is also worth noting that the single run I mentioned before ran successfully but didn’t return any data (couldn’t retrieve it in grasshopper).

I noticed some errors with the GH components. There was the time-out error but that has been explained and addressed. There have also been some other errors but they have since been fixed and mitigated.

I’ll send some new combinations off soon. What usually has the potentially to increase file upload times and file sizes; geometry or data?

I did notice that the simulation time for daylight (which uses the same script and model etc) took ages to complete - 3 days or there abouts.

Thanks! That will help. I’ll also take a look at the other jobs that succeeded and failed and see if I can dig anything else up.

I’m not sure re: HBJSON file size. I would think geometry, but @mostapha would know more than me. The number of them is definitely part of it too. They are each fairly small (~1.5MB) but with hundreds of them, that’s a lot of upload requests.

Wrt the simulation times, I’m not sure those are totally trustworthy right now. A lot of the ones you submitted were ones that had missing update events and we pushed to completed manually. Looking at the actual workflows, I don’t see anything that took longer than 12 hours.

1 Like