Choosing fields

[paquino11]

Hi!

First of all, thank you very much for creating this amazing repo.

I was wondering if it is possible to choose just the fields we want to scrape instead of scraping the entire list available.

Thanks!

[gosom]

@paquino11 choosing the fields to scrape will require significant modifications that do not worth the effort.

Do you want to improve perfomance or you just want the output to have only the fields you want?

[paquino11]

Hi,

Mainly performance and resources since I just need 4-ish fields.

I was trying to modify the entry.go file by commenting fields I don't need on EntryFromJSON and CsvRow functions and Entry type, but didn't work.

Are the changes done inside the Docker file? I can support you on that if you don't mind.

[gosom]

To do it properly you need to pass as parameters the fields you need and then based on that dynamically execute only the relevant code.
Probably also the Job's properties need to be mdified to be pointers and if a value is nil needs to be excluded for the CSV.

Regarding your questions I don't fully understand this. If you change something locally then you need to build another docker image .

a Pull request is more than welcome @paquino11 .

In any case I will try to add this feature since it was requested also by others.