enhancement: store blocks while crawling transactions and chain

[joel-u410]

No description provided.

[joel-u410]

Working on fixing tests here -- it's failing because in this branch there is now a foreign key constraint between balance_changes --> blocks and the tests need to insert blocks before inserting balances.

[mateuszjasiuk]

@joel-u410 hello, can you point this to 1.0.0-maint ?

[joel-u410]

@joel-u410 hello, can you point this to 1.0.0-maint ?

Yep, will do!

[joel-u410]

So, in #203 I have added code that resolves tm_addr -> namada_addr for block proposers.

I'm wondering now if we should do the same here, and store the namada_address for block proposers into the blocks table instead of the tendermint address as it's currently doing?

@mateuszjasiuk @Fraccaman what do you think?

[Fraccaman]

Uhmmm so, in future we will need to have a fk to the validator table, so we can join block and validator. Maybe this is something we can do now too? Or do you think it would be too much?

[joel-u410]

Uhmmm so, in future we will need to have a fk to the validator table, so we can join block and validator. Maybe this is something we can do now too? Or do you think it would be too much?

That makes sense, especially if blocks.proposer_address were storing the namada_address of the validator. (which it isn't, yet)

Although, I would be concerned about failing to insert a block if its validator hadn't previously been indexed. I suppose we could insert a stubbed record in that case, to keep the fk happy.

I'd like to rebase this one & revisit the fk idea after #203 is merged, since then we will easily have the namada_address available for the proposer in each block and I can switch this to storing that instead of the tm_addr.

[mateuszjasiuk]

Cool! I'd prefer to merge this sooner than later as this is needed for shieled sync improvements in namadillo, so maybe let's do fk thing later.

Edit:
I tried to test it, but I had some compilation errors. I think it's because you forgot to regenerate diesel schema(the fields in block should be nullable except for height). There was also a problem with initial query for empty db, I think we have to seed db with empty blocks first. With those changes it seems to work fine, but please check if it makes sense :D

[joel-u410]

@mateuszjasiuk I'll take a look today

[joel-u410]

Edit: I tried to test it, but I had some compilation errors. I think it's because you forgot to regenerate diesel schema(the fields in block should be nullable except for height).

Really weird, I didn't have any issues compiling. But makes sense though -- I did change the schema to make those columns nullable, so the diesel schema should indeed be regenerated.

Edit: I see now -- I didn't need to update the diesel schema with Nullable because in my branch I was never actually inserting a block with any null values, however there could certainly be a problem with querying blocks that contain nulls, which I never did on my branch.

There was also a problem with initial query for empty db, I think we have to seed db with empty blocks first.

Hmm .. actually it's probably better in that case to query the actual initial block instead of seeding with an empty block. I'll take a closer look.

[joel-u410]

ok @mateuszjasiuk please test again, it should be good to merge now, I made the following changes:

• Rebased to latest 1.0.0-maint
• Incorporated your updates to the diesel schema
• During initial_query, fetches & upserts the actual block instead of a blank one
• Uses the namada_address for block.proposer instead of the tendermint address

[mateuszjasiuk]

Nice! This works. Thanks for quick response :)