v0.7.0 - Model Downloader

[saddam213]

Model Downloader

Trying to simplify the experience for newbies, have added a simplified way to add/import/export models into the UI

This is mainly a UI release, with a few improvements and bug fixes

What's Changed

• Model Downloader/Manager
• Support multiple timestep input types
• add solution without dependencies on example projects by @james-s-tayler in add solution without dependencies on example projects #13
• Added model specific tokens to the model template descriptions of the models that have them. by @TheyCallMeHex in Added model specific tokens to the model template descriptions of the models that have them. #17

New Contributors

• @james-s-tayler made their first contribution in add solution without dependencies on example projects #13

Full Changelog: v0.6.0...v0.7.0

---

This discussion was created from the release v0.7.0 - Model Downloader.

[Amin456789]

hey Adam! thank u for this lovely update! the ui looks so beautiful and i can see all buttons now! lcm fp16 works very very good and it is so fast! thank u a lot! it is a huge update to me!

EDIT: the features i asked are not that neseccery so i removed them, with a quick bat i made that can take me to the output folder!

thank u again guys!

[Amin456789]

btw Adam, u should overhaul ur description page and add all these new features that u made in this time! like putting screen shots of the gui and all it can do! lots of people will love it

btw, i asked another onnx developer about animatediff in the past, i told him that u r working on it too, u guys can help each other in writing the pipeline i think. @ssube please reach out here

[patientx]

I have models previously created with amd's guide BUT it seems one thing is different , in unet folder of one of the models I downloaded from the list, there are two files model.onnx which is a small file and model.onnx_data which is a large file, with my locally converted models I only have one large model.onnx so app gives error and can't seem to work with it. Is there any way to salvage these models? They work with amd fork of a1111 's olive mode , and I'd like to be able to use them here as well if possible. Or can you point out an "easier" way to convert models to onnx locally ?

[saddam213]

some unet models have multiple files, some have one, The app only needs the model.onnx, as that file knows about the rest, you just have to reference the model.onnx file

If you are making model template.json to share then you will need to put all file in the download file list though

[tonual]

Great idea to automate the model downloader!

Could you please give me a hint what could went wrong with inferencing the LCM-Dreamshaper-V7?
I have downloaded models from:
https://huggingface.co/TheyCallMeHex/LCM-Dreamshaper-V7-ONNX/tree/main

so that my sd_dreamshaper folder looks like this:

rk@Rafals-Mac-mini sd_dreamshaper % pwd
/Users/rk/Documents/repos/onnx/sd_dreamshaper
rk@Rafals-Mac-mini sd_dreamshaper % ls
cliptokenizer.onnx textencoder.onnx unet.onnx unet.onnx_data vae_dec.onnx vae_enc.onnx
rk@Rafals-Mac-mini sd_dreamshaper %

and JSON config:

{
"Name": "LCM-Dreamshaper-V7",
"IsEnabled": true,
"PadTokenId": 49407,
"BlankTokenId": 49407,
"TokenizerLimit": 77,
"EmbeddingsLength": 768,
"ScaleFactor": 0.18215,
"PipelineType": "LatentConsistency",
"Diffusers": [
"TextToImage",
"ImageToImage"
],
"DeviceId": 0,
"InterOpNumThreads": 0,
"IntraOpNumThreads": 0,
"ExecutionMode": "ORT_SEQUENTIAL",
"ExecutionProvider": "CoreML",
"ModelConfigurations": [
{
"Type": "Tokenizer",
"OnnxModelPath": "/Users/rk/Documents/repos/onnx/sd_dreamshaper/cliptokenizer.onnx"
},
{
"Type": "Unet",
"OnnxModelPath": "/Users/rk/Documents/repos/onnx/sd_dreamshaper/unet.onnx"
},
{
"Type": "TextEncoder",
"OnnxModelPath": "/Users/rk/Documents/repos/onnx/sd_dreamshaper/textencoder.onnx"
},
{
"Type": "VaeEncoder",
"OnnxModelPath": "/Users/rk/Documents/repos/onnx/sd_dreamshaper/vae_enc.onnx"
},
{
"Type": "VaeDecoder",
"OnnxModelPath": "/Users/rk/Documents/repos/onnx/sd_dreamshaper/vae_dec.onnx"
}
]
}

However, I got this error while inferencing the model:

Loading Model LCM-Dreamshaper-V7...
2023-11-15 13:11:46.392 OnnxStack.Console[1528:23845] 2023-11-15 13:11:46.393511 [W:onnxruntime:, coreml_execution_provider.cc:81 GetCapability] CoreMLExecutionProvider::GetCapability, number of partitions supported by CoreML: 85 number of nodes in the graph: 4230 number of nodes supported by CoreML: 184
2023-11-15 13:11:47.182 OnnxStack.Console[1528:23845] 2023-11-15 13:11:47.183283 [W:onnxruntime:, session_state.cc:1162 VerifyEachNodeIsAssignedToAnEp] Some nodes were not assigned to the preferred execution providers which may or may not have an negative impact on performance. e.g. ORT explicitly assigns shape related ops to CPU to improve perf.
2023-11-15 13:11:47.182 OnnxStack.Console[1528:23845] 2023-11-15 13:11:47.183325 [W:onnxruntime:, session_state.cc:1164 VerifyEachNodeIsAssignedToAnEp] Rerunning with verbose output on a non-minimal build will show node assignments.
Unhandled exception. Microsoft.ML.OnnxRuntime.OnnxRuntimeException: [ErrorCode:Fail] Deserialize tensor up_blocks.2.resnets.2.time_emb_proj.bias failed.GetFileLength for /Users/rk/Documents/repos/onnx/sd_dreamshaper/model.onnx_data failed:Invalid fd was supplied: -1
at Microsoft.ML.OnnxRuntime.InferenceSession.Init(String modelPath, SessionOptions options, PrePackedWeightsContainer prepackedWeightsContainer)
at Microsoft.ML.OnnxRuntime.InferenceSession..ctor(String modelPath, SessionOptions options, PrePackedWeightsContainer prepackedWeightsContainer)
at OnnxStack.Core.Model.OnnxModelSession..ctor(OnnxModelSessionConfig configuration, PrePackedWeightsContainer container) in /Users/rk/Documents/repos/OnnxStack/OnnxStack.Core/Model/OnnxModelSession.cs:line 29
at OnnxStack.Core.Model.OnnxModelSet.<.ctor>b__3_1(OnnxModelSessionConfig modelConfig) in /Users/rk/Documents/repos/OnnxStack/OnnxStack.Core/Model/OnnxModelSet.cs:line 25
at System.Collections.Immutable.ImmutableDictionary.<>c__DisplayClass9_03.<ToImmutableDictionary>b__0(TSource element) at System.Linq.Enumerable.SelectListIterator2.MoveNext()
at System.Collections.Immutable.ImmutableDictionary2.AddRange(IEnumerable1 items, MutationInput origin, KeyCollisionBehavior collisionBehavior)
at System.Collections.Immutable.ImmutableDictionary2.AddRange(IEnumerable1 pairs, Boolean avoidToHashMap)
at System.Collections.Immutable.ImmutableDictionary2.AddRange(IEnumerable1 pairs)
at System.Collections.Immutable.ImmutableDictionary.ToImmutableDictionary[TSource,TKey,TValue](IEnumerable1 source, Func2 keySelector, Func2 elementSelector, IEqualityComparer1 keyComparer, IEqualityComparer1 valueComparer) at System.Collections.Immutable.ImmutableDictionary.ToImmutableDictionary[TSource,TKey,TValue](IEnumerable1 source, Func2 keySelector, Func2 elementSelector)
at OnnxStack.Core.Model.OnnxModelSet..ctor(IOnnxModelSetConfig configuration) in /Users/rk/Documents/repos/OnnxStack/OnnxStack.Core/Model/OnnxModelSet.cs:line 23
at OnnxStack.Core.Services.OnnxModelService.LoadModelSet(IOnnxModel model) in /Users/rk/Documents/repos/OnnxStack/OnnxStack.Core/Services/OnnxModelService.cs:line 302
at OnnxStack.Core.Services.OnnxModelService.<>c__DisplayClass5_0.b__0() in /Users/rk/Documents/repos/OnnxStack/OnnxStack.Core/Services/OnnxModelService.cs:line 53
at System.Threading.Tasks.Task`1.InnerInvoke()
at System.Threading.ExecutionContext.RunFromThreadPoolDispatchLoop(Thread threadPoolThread, ExecutionContext executionContext, ContextCallback callback, Object state)
--- End of stack trace from previous location ---
at System.Threading.ExecutionContext.RunFromThreadPoolDispatchLoop(Thread threadPoolThread, ExecutionContext executionContext, ContextCallback callback, Object state)
at System.Threading.Tasks.Task.ExecuteWithThreadLocal(Task& currentTaskSlot, Thread threadPoolThread)
--- End of stack trace from previous location ---
at OnnxStack.Core.Services.OnnxModelService.LoadModel(IOnnxModel model) in /Users/rk/Documents/repos/OnnxStack/OnnxStack.Core/Services/OnnxModelService.cs:line 53
at OnnxStack.StableDiffusion.Services.StableDiffusionService.LoadModel(IModelOptions modelOptions) in /Users/rk/Documents/repos/OnnxStack/OnnxStack.StableDiffusion/Services/StableDiffusionService.cs:line 56
at OnnxStack.Console.Runner.RafStableDiffusion.RunAsync() in /Users/rk/Documents/repos/OnnxStack/OnnxStack.Console/Examples/RafStableDiffusion.cs:line 59
at OnnxStack.Console.AppService.StartAsync(CancellationToken cancellationToken) in /Users/rk/Documents/repos/OnnxStack/OnnxStack.Console/AppService.cs:line 19
at Microsoft.Extensions.Hosting.Internal.Host.StartAsync(CancellationToken cancellationToken)
at Microsoft.Extensions.Hosting.HostingAbstractionsHostExtensions.RunAsync(IHost host, CancellationToken token)
at Microsoft.Extensions.Hosting.HostingAbstractionsHostExtensions.RunAsync(IHost host, CancellationToken token)
at OnnxStack.Console.Program.Main(String[] _) in /Users/rk/Documents/repos/OnnxStack/OnnxStack.Console/Program.cs:line 27
at OnnxStack.Console.Program.

(String[] _)

[saddam213]

If you are using the LCM model you can only use the LCM Scheduler

Try setting SchedulerType = SchedulerType.LCM in you scheduler options, that should resolve that exception

[tonual]

Thanks a lot, it works.

[saddam213]

DiffuserPipelineType has a handy extension method GetSchedulerTypes that will return the compatible SchedulerType for each pipeline

var schedulers = model.PipelineType.GetSchedulerTypes()

[tonual]

My suggestion would be to have all those constraints in the model so that SD noobies won't hurt themselves that easily.

[saddam213]

Multiple pipelines is still new and under construction, it will get better over time