shovel first time user experience report

[psugihara]

Hey @ryandotsmith, this project is really cool. Thanks for making it.

I thought it might be useful to share a few notes on rough edges from a first-time user and newish blockchain person. I used shovel via the @indexsupply/shovel-config ts package:

• log_addr, etc — would be so nice to have the table in the docs explain what these are / where they come from. I know it’s probably out of scope to like explain blockchains to me, but just giving you one data point that I was learning all of these terms via shovel and wasn’t sure what was shovel-specific or not.
• could types be inferred for more stuff? Built in type mapper would be nice. It seems like certain types must be mapped a certain way to avoid errors (e.g. address to bytea), so not sure if having me specify the column type is really necessary. I wrote a script that creates integrations for a given ABI. Seems like that could be generalized and built in.
• gotcha: Column names must be lowercase or you get unrelated looking errors. Took a lot of random experimentation to figure out what the problem was with this. I'm coming from prisma where it's common to do camel casing.
• dashboard
  • so cool!
  • row count usually shows 0 even though there’s stuff in the db. Maybe it’s supposed to show real time processing stuff maybe or maybe it’s a bug, unclear. Either way, showing total rows on the dashboard seems useful.
  • +Source, +Integration… couldn’t figure out what these do

No need to respond directly to any of this but I value this kind of brain dump from a user when I'm building, so thought I'd pass along.

All best,
Peter

[ryandotsmith]

Excellent feedback. Thank you.

• log_addr, etc — would be so nice to have the table in the docs explain what these are / where they come from. I know it’s probably out of scope to like explain blockchains to me, but just giving you one data point that I was learning all of these terms via shovel and wasn’t sure what was shovel-specific or not.

I assume you've seen this and are looking for more?

https://indexsupply.com/shovel/docs/#data

It's possible that we could link out to some Ethereum docs here. Not sure what page would be the best to link to.

• could types be inferred for more stuff? Built in type mapper would be nice. It seems like certain types must be mapped a certain way to avoid errors (e.g. address to bytea), so not sure if having me specify the column type is really necessary. I wrote a script that creates integrations for a given ABI. Seems like that could be generalized and built in.

Certainly. The TS package could do a whole lot more. Please share any code that you think might be useful for others. Perhaps we can even merge it into the package?

• gotcha: Column names must be lowercase or you get unrelated looking errors. Took a lot of random experimentation to figure out what the problem was with this. I'm coming from prisma where it's common to do camel casing.

Yes. That could probably be detected in the config parsing so that we can return an error. It is documented here:
https://indexsupply.com/shovel/docs/#config-integrations-table-columns-name

OOC did you find this in the docs?

• dashboard

  • so cool!
  • row count usually shows 0 even though there’s stuff in the db. Maybe it’s supposed to show real time processing stuff maybe or maybe it’s a bug, unclear. Either way, showing total rows on the dashboard seems useful.
  • +Source, +Integration… couldn’t figure out what these do

The dashboard needs some work...

[psugihara]

np :) here's the ts config generator I ended up with if it's useful to anyone else: https://github.com/trifle-labs/anybody-problem/blob/09a4053f1477439986ab0185ca8d9a1b56fb28ef/server/shovel-config.ts

I assume you've seen this and are looking for more?

Yes, did find that just didn't know what those were but hope I mostly figured it out. I definitely looked at ethereum docs too. Some pointers in the table or like 1 line descriptions would help immensely. The examples were helpful to figure out that I was supposed to use log_addr to address the contract but when I googled "log_addr ethereum" there's nothing relevant looking except for shovel docs.

OOC did you find this in the docs?

No, missed that whoops!

That could probably be detected in the config parsing so that we can return an error

yes!! great

[psugihara]

Also some things I think are great:

• 1 page docs that are easy to grep
• scope of tool and JSON config interface
• general vibe and performance
• price