Make PyArrow datetime timezone test pass #5014

gvwilson · 2025-02-06T18:35:31Z

plotly/tests/test_optional/test_px/test_px_hover.py contains a test test_date_in_hover that currently marks the PyArrow case xfail because of problems with the constructor. We need to fix that test.

@FBruzzesi writes:

My understanding is that you would like to build the table directly with a "date" being datetime type.
Not sure which kind of conversions are possible in the initialization - especially, what we use from string to datetime (pyarrow.compute.strptime) requires an explicit format, so I would be surprised if only during initialization this is inferred.

What you can do is to pass a datetime object:

from datetime import datetime
from zoneinfo import ZoneInfo
import pyarrow as pa

data = {
    'date': [datetime.datetime(2015, 4, 4, 19, 31, 30, tzinfo=zoneinfo.ZoneInfo(key='Europe/Rome'))],
    'value': [3]
}

schema = pa.schema([("date", pa.timestamp(unit="ms", tz="Europe/Berlin")), ("value", pa.int64())])
pa.table(data, schema=schema)

pyarrow.Table
date: timestamp[ms, tz=Europe/Berlin]
value: int64
----
date: [[2015-04-04 17:31:30.000Z]]
value: [[3]]

In narwhals you could do the same:

import narwhals as nw

schema = {
    "date": nw.Datetime(time_zone="Europe/Rome", time_unit="ms"),
    "value": nw.Int64()
}

nw.from_dict(data, schema, native_namespace=pa)
┌───────────────────────────────────┐
|        Narwhals DataFrame         |
|-----------------------------------|
|pyarrow.Table                      |
|date: timestamp[ms, tz=Europe/Rome]|
|value: int64                       |
|----                               |
|date: [[2015-04-04 17:31:30.000Z]] |
|value: [[3]]                       |
└───────────────────────────────────┘

The text was updated successfully, but these errors were encountered:

FBruzzesi · 2025-02-06T19:26:36Z

Hey @gvwilson , thanks for the ping.
I might be missing something but the test is not marked to xfail for pyarrow at the moment and I don't see it failing either in your CI (at least for pyarrow).
Similarly, locally I don't get any issue.

Just to make sure, this is the test:

plotly.py/plotly/tests/test_optional/test_px/test_px_hover.py

Lines 187 to 196 in 86a7e12

    
           def test_date_in_hover(constructor): 
        
               df = nw.from_native( 
        
                   constructor({"date": ["2015-04-04 19:31:30+01:00"], "value": [3]}) 
        
               ).with_columns(date=nw.col("date").str.to_datetime(format="%Y-%m-%d %H:%M:%S%z")) 
        
               fig = px.scatter(df.to_native(), x="value", y="value", hover_data=["date"]) 
        
               # Check that what gets displayed is the local datetime 
        
               assert nw.to_py_scalar(fig.data[0].customdata[0][0]) == nw.to_py_scalar( 
        
                   df.item(row=0, column="date") 
        
               ).replace(tzinfo=None)

gvwilson added bug something broken P1 needed for current cycle testing automated tests labels Feb 6, 2025

gvwilson self-assigned this Feb 6, 2025

gvwilson closed this as completed in c58e73a Feb 7, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

Make PyArrow datetime timezone test pass #5014

Make PyArrow datetime timezone test pass #5014

Uh oh!

Uh oh!

Make PyArrow datetime timezone test pass #5014

Make PyArrow datetime timezone test pass #5014

Comments

Uh oh!

Uh oh!