Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

docs: Add to supported data types table #2192

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

NickCrews
Copy link
Contributor

I was getting confused as to how my pyarrow timestamps
were getting converted, and I think seeing this here would have
pointed me in the right direction sooner.

I was getting confused as to how my pyarrow timestamps
were getting converted, and I think seeing this here would have
pointed me in the right direction sooner.
@JovanVeljanoski
Copy link
Member

About the timestamp array in arrow.. so in vaex i believe it is stored as arrow, it is not really converted to numpy. However, since many of the datetime and timedelta operations supported come via pandas, it is true that on the fly the data is cast to numpy (well pandas series).

But the data is still stored as arrow.

I don't know how to state this exactly in the short text for the purpose of that table so what you've written probably is fine.. @maartenbreddels ?

@maartenbreddels
Copy link
Member

Indeed, it's being converted on the fly when doing many operations. How did it confused you @NickCrews ?

@NickCrews
Copy link
Contributor Author

I don't remember exactly now 😭, should have been more detailed.

But I think it was constructing a DF from a pyarrow timestamp array, doing some operations on it, and then re-exporting with .values, and I was surprised that it had turned into a numpy array.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants