8000 unix/modtime: Add mktime function. by andrewleech · Pull Request #5333 · micropython/micropython · GitHub
[go: up one dir, main page]

Skip to content

unix/modtime: Add mktime function. #5333

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 1 commit into from

Conversation

andrewleech
Copy link
Contributor

Add utime.mktime() to match other ports.

Function implementation was copied from stm32 version.

@dpgeorge
Copy link
Member

There's a bit of history regarding the lack of this function, see #4576 and #4583. Will need to review the arguments for/against the inclusion of this.

@andrewleech
Copy link
Contributor Author

I missed those issues. The argument seems to be that utime doesn't match cpython anyway and unix doesn't have a RTC so shouldn't have this function?

I've got existing stm32 code that uses mktime to convert between tuple based time and seconds based timestamps. mktime is pretty standalone in its usage to do this.

Perhaps when the previous arguments were made unix didn't have utime.localtime() to provide time in tuple format.
It does now at least, so the ability to convert between tuple and seconds seems like a pretty natural match.

@dpgeorge
Copy link
Member

Perhaps when the previous arguments were made unix didn't have utime.localtime() to provide time in tuple format.

Hmm, I don't think so, localtime() was added in Jun 2017 in d42b80f

@andrewleech
Copy link
Contributor Author

Sure, I don't really follow the counter-argument then, not sure what the issue was.
mktime is implemented in a common lib already and doesn't rely on any hardware etc.

@@ -161,6 +162,29 @@ STATIC mp_obj_t mod_time_localtime(size_t n_args, const mp_obj_t *args) {
}
STATIC MP_DEFINE_CONST_FUN_OBJ_VAR_BETWEEN(mod_time_localtime_obj, 0, 1, mod_time_localtime);

/// \function mktime()
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minor, but probably better to use the same style as the rest of the code: no need to repeat function name, just 2 //. Likewise for the code, I don't feel it needs all those newlines or at least not the first one in the function body.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Valid point, I updated the function name to match this files style but didn't pay attention to any of the other styling.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This commenting style is really obsolete. I intend to delete it all from stm32 at some point...

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think I've cleaned up the styling to match the rest of the file

@stinos
Copy link
Contributor
stinos commented Nov 15, 2019

I'm +1 to including this, main reason that if localtime is available, mktime should be as well I guess. However it feels a bit silly to just copy the function once more, couldn't all ports providing this use the same one?

@andrewleech
Copy link
Contributor Author

@stinos I tend to agree that consolidating the wrapper functionality here would be nice, though there's only some parts on this file that can easily be done, others are certainly port specific.

The bulk of the code is already consolidated in lib/timeutils, it's just the python object wrapping that's separated in mktime here. Right now I don't know of an existing common file that makes sense to move just mktime object handling to.
I think a broader consolidation effort should be done as a standalone task, potentially in line with #4821 (comment)

@stinos
Copy link
Contributor
stinos commented Nov 18, 2019

Yeah maybe it's over-engineering in this case, but it's just that I looked up mktime throughout the entire code and saw that 'CPython uses 9 etc' logic 5 times or so, and I'm a fan of deduplication just because I've been screwed by duplication so many times in the past. But indeed I'm not sure if it's worth it here and it could be done separately.

@andrewleech
Copy link
Contributor Author

Hehe I'm all too often the same, but then I get tunnel vision and refactor everything as I go and end up with something like https://github.com/micropython/micropython/pull/5249/files with 37 files changed.

Smaller self-contained changes are best, ideally with discussion leading to logical next steps.

@andrewleech
Copy link
Contributor Author

Ok, so there's probably a problem with this PR regarding epoch. In unix port, localtime() works on an epoch of 1970 but the micropython\lib\timeutils\timeutils.c implementation of mktime that this PR uses does the conversion the same as other mpy ports, to 2000.

So for consistency, I probably need to update timeutils to support the older epoch for linux port to ensure consistency with localtime.

Alternatively and/or separately, we look at fixing epoch consistency across all the ports?

@dpgeorge
Copy link
Member

So for consistency, I probably need to update timeutils to support the older epoch for linux port to ensure consistency with localtime.

Yes. It should be that time.mktime(time.localtime()) is the same as time.time()

Alternatively and/or separately, we look at fixing epoch consistency across all the ports?

The uPy-specific 2000 epoch does lead to difficulties when using it, as people expect the 1970 epoch. But it's hard to solve this. Ideally all ports would use the 1970 epoch but I don't see how to do it. One option is to provide a constant like utime.EPOCH_START which converts to 1970, ie utime.time() + utime.EPOCH_START is the number of seconds since 1970. On unix EPOCH_START is just 0.

@andrewleech
Copy link
Contributor Author

Yes. It should be that time.mktime(time.localtime()) is the same as time.time()

So I started adding a "seconds from (2000 - 1970)" offset in the unix port, it's still not really enough to satisfy this expectation: utime.time() is from epoch, localtime() is offset by timezone/dst.

@dpgeorge
Copy link
Member

utime.time() is from epoch, localtime() is offset by timezone/dst.

I see. So time.mktime() (in CPython) expects its input to be in local time, and adjusts accordingly (because Epoch time is absolute and can't depend on local time offset).

It's insightful to look at the result of time.mktime(time.gmtime(0)). When you're at GMT+0, time.gmtime(0) is the Epoch and this expression returns 0. But at GMT+10, time.gmtime(10 * 3600) is the Epoch, so the expression returns -36000.

So a mktime implementation needs to subtract the current timezone. Probably easiest to just use the built-in mktime (from C stdlib).

@andrewleech
Copy link
Contributor Author

Yeah true, utime.localtime function is calling through to C localtime, makes sense for time.mktime to do the same.

@andrewleech
Copy link
Contributor Author

Ok, new version using C mktime pushed. time.mktime(time.localtime(0)) == 0

@@ -29,6 +29,7 @@ endif
# source files
SRC_C = \
lib/utils/printf.c \
lib/timeutils/timeutils.c \
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shouldn't need this anymore...?

// printf("sec: %d\n", (int)time.tm_sec);
// printf("wday: %d\n", (int)time.tm_wday);
// printf("yday: %d\n", (int)time.tm_yday);
// printf("isdst: %d\n", (int)time.tm_isdst);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please remove debugging code.

@@ -126,6 +130,8 @@ STATIC mp_obj_t mod_time_sleep(mp_obj_t arg) {
}
STATIC MP_DEFINE_CONST_FUN_OBJ_1(mod_time_sleep_obj, mod_time_sleep);



Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please remove blank lines.

@@ -66,6 +67,9 @@ static inline int msec_sleep_tv(struct timeval *tv) {
#error Unsupported clock() implementation
#endif

#define DAYS_PER_4Y (365*4 + 1)
#define SECONDS_1970_2000 ((DAYS_PER_4Y * (2000-1970) / 4) * 24 * 60 * 60)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are these needed?


// localtime generates a tuple of len 8. CPython uses 9, so we accept both.
if (len < 8 || len > 9) {
nlr_raise(mp_obj_new_exception_msg_varg(&mp_type_TypeError, "mktime needs a tuple of length 8 or 9 (%d given)", len));
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Don't need to say how many items were given, just a simple error msg would be enough (using mp_raise_TypeError).

@pi-anl pi-anl force-pushed the unix_mktime branch 3 times, most recently from 710b1c6 to 3de7362 Compare November 27, 2019 04:38
@andrewleech
Copy link
Contributor Author

Hmm, my test doesn't work on the unix nanbox build, not sure why.

@dpgeorge
Copy link
Member

Hmm, my test doesn't work on the unix nanbox build, not sure why.

It looks like you ran into the Year 2038 problem: https://en.wikipedia.org/wiki/Year_2038_problem

Nanbox builds are 32-bit, and mktime() thus returns a 32-bit signed number, which can't represent anything beyond 2037. So it's returning -1 instead. Probably need to catch such a case and raise an exception... (not sure what CPython 32-bit would do here).

@andrewleech
Copy link
Contributor Author

Ah, that makes sense. It looks like cpython has quite a bit of extra parsing and error checking going on, some of which appears to be for this:
https://github.com/python/cpython/blob/master/Modules/timemodule.c#L955

@dpgeorge
Copy link
Member

Looks like OverflowError is the right thing to throw then.

@dpgeorge
Copy link
Member

This will need to pass CI before it can be merged.

@andrewleech
Copy link
Contributor Author

Yes, there's the tests that break due to the date overflow issue and the exception being thrown. I haven't decided whether to try to handle that properly on ports where expected of just restrict the date rage to safe dates...

Also applies to windows port.
@andrewleech
Copy link
Contributor Author

I'm not sure why the AppVeyor buiild is failing, the log I looked at showed the build compiled fine and all the tests were either pass or skipped.

@dpgeorge
Copy link
Member

I'm not sure why the AppVeyor buiild is failing, the log I looked at showed the build compiled fine and all the tests were either pass or skipped.

I pushed this PR as a new PR in #5461 to try and fix this.

@dpgeorge
Copy link
Member

Merged in 1b844e9 (via #5461).

@dpgeorge dpgeorge closed this Dec 28, 2019
tannewt pushed a commit to tannewt/circuitpython that referenced this pull request Sep 15, 2021
Make `next_code_allocation` and `prev_traceback_allocation` movable
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants
0