8000 Datetime fix for MinGW. by 87 · Pull Request #214 · numpy/numpy · GitHub
[go: up one dir, main page]
Skip to content

Datetime fix for MinGW. #214

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
wants to merge 6 commits into from
Closed

Datetime fix for MinGW. #214

wants to merge 6 commits into from

Conversation

87
Copy link
Contributor
@87 87 commented Feb 20, 2012

Hi,

This should fix the 32/64 bits issues with the time type when using MinGW. (See also here)

@rgommers
Copy link
Member

It fixes all the failures with MinGW + python 2.5, with python 3.1 a few issues are left. This is with PR-213 and this one together.

======================================================================
ERROR: test_datetime_y2038 (test_datetime.TestDateTime)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "Z:\Users\rgommers\.wine\drive_c\Python31\lib\site-packages\numpy\core\tests\test_datetime.py", line 1706, in test_datetime_y2038
    a = np.datetime64('2038-01-20T13:21:14')
OSError: Failed to use mktime to convert local time to UTC

======================================================================
ERROR: test_maskna.test_array_maskna_astype
----------------------------------------------------------------------
Traceback (most recent call last):
  File "Z:\Users\rgommers\.wine\drive_c\Python31\lib\site-packages\nose-3.0.0.dev-py3.1.egg\nose\case.py", line 178, in runTest
    self.test(*self.arg)
  File "Z:\Users\rgommers\.wine\drive_c\Python31\lib\site-packages\numpy\core\tests\test_maskna.py", line 212, in test_array_maskna_astype
    b = a.astype(dt2)
TypeError: complex() argument must be a string or a number

======================================================================
ERROR: Failure: OSError (Failed to use mktime to convert local time to UTC)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "Z:\Users\rgommers\.wine\drive_c\Python31\lib\site-packages\nose-3.0.0.dev-py3.1.egg\nose\failure.py", line 37, in runTest
    reraise(self.exc_class, self.exc_val, self.tb)
  File "Z:\Users\rgommers\.wine\drive_c\Python31\lib\site-packages\nose-3.0.0.dev-py3.1.egg\nose\_3.py", line 7, in reraise
    raise exc_class(exc_val).with_traceback(tb)
  File "Z:\Users\rgommers\.wine\drive_c\Python31\lib\site-packages\nose-3.0.0.dev-py3.1.egg\nose\loader.py", line 389, in loadTestsFromName
    addr.filename, addr.module)
  File "Z:\Users\rgommers\.wine\drive_c\Python31\lib\site-packages\nose-3.0.0.dev-py3.1.egg\nose\importer.py", line 39, in importFromPath
    return self.importFromDir(dir_path, fqname)
  File "Z:\Users\rgommers\.wine\drive_c\Python31\lib\site-packages\nose-3.0.0.dev-py3.1.egg\nose\importer.py", line 86, in importFromDir
    mod = load_module(part_fqname, fh, filename, desc)
  File "Z:\Users\rgommers\.wine\drive_c\Python31\lib\site-packages\numpy\core\tests\test_multiarray.py", line 915, in <module>
    class TestArgmax(TestCase):
  File "Z:\Users\rgommers\.wine\drive_c\Python31\lib\site-packages\numpy\core\tests\test_multiarray.py", line 941, in TestArgmax
    np.datetime64('2041-12-03T14:05:03')], 5),
OSError: Failed to use mktime to convert local time to UTC

@mwiebe
Copy link
Member
mwiebe commented Feb 20, 2012

My guess would be that it's somehow not triggering the 64-bit time_t.

@87
Copy link
Contributor Author
87 commented Feb 20, 2012

Hmm, maybe mktime needs to be _mktime64 for the same reasons. I don't get why it works for Python 2.5 though..

@rgommers
Copy link
Member

The errors on 2.5 were never the same ones as on 2.7/3.1. Perhaps related to the Python 2.5 installers being built with VS 2005 (IIRC) and 2.7/3.1 with VS 2008.

@87
Copy link
Contributor Author
87 commented Feb 20, 2012

Ralf, if you change mktime on this line to _mktime64, will it fix the issue on Python 3.1?

@87
Copy link
Contributor Author
87 commented Feb 20, 2012

Ah, I think Python 2.5 might even have been compiled with VS 2003, if I remember correctly.

@87
Copy link
Contributor Author
87 commented Feb 20, 2012

I think I understand why Python 2.5 succeeds. It doesn't use the custom functionality, and therefore falls back on a 32-bits time_t and -time functionality. Because the datetime code makes sure that 32-bits time_t size is handled with a kind of overflow mechanism for dates beyond 2038, it does not fail in those cases..

@rgommers
Copy link
Member

Han, after changing mktime to _mktime64 as you suggest, the same tests are still failing. Just different errors than before:

======================================================================
ERROR: test_datetime_y2038 (test_datetime.TestDateTime)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "Z:\Users\rgommers\.wine\drive_c\Python31\lib\site-packages\numpy\core\tests\test_datetime.py", line 1707, in test_datetime_y2038
    assert_equal(str(a)[:-5], '2038-01-20T13:21:14')
OSError: Failed to use '_localtime64_s' to convert to a local time

======================================================================
ERROR: test_maskna.test_array_maskna_astype
----------------------------------------------------------------------
Traceback (most recent call last):
  File "Z:\Users\rgommers\.wine\drive_c\Python31\lib\site-packages\nose-3.0.0.dev-py3.1.egg\nose\case.py", line 178, in runTest
    self.test(*self.arg)
  File "Z:\Users\rgommers\.wine\drive_c\Python31\lib\site-packages\numpy\core\tests\test_maskna.py", line 212, in test_array_maskna_astype
    b = a.astype(dt2)
TypeError: complex() argument must be a string or a number

======================================================================
ERROR: test_combinations (test_multiarray.TestArgmax)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "Z:\Users\rgommers\.wine\drive_c\Python31\lib\site-packages\numpy\core\tests\test_multiarray.py", line 979, in test_combinations
    assert_equal(np.argmax(arr), pos, err_msg="%r"%arr)
OSError: Failed to use '_localtime64_s' to convert to a local time

======================================================================
ERROR: test_combinations (test_multiarray.TestArgmin)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "Z:\Users\rgommers\.wine\drive_c\Python31\lib\site-packages\numpy\core\tests\test_multiarray.py", line 1047, in test_combinations
    assert_equal(np.argmin(arr), pos, err_msg="%r"%arr)
OSError: Failed to use '_localtime64_s' to convert to a local time

@87
Copy link
Contributor Author
87 commented Feb 20, 2012

That is strange, have you applied all patches here? This seems to be the same bug we had before (with a 32-bits time_t), but that should have been fixed.

@rgommers
Copy link
Member

Yes, PR-213, then this PR, then the one-line change rawtime = _mktime64(&tm_); in `datetime_strings.c.

The difference with the 2.5 errors from before is that those were using localtime and 3.1 is now failing on _localtime64_s.

@rgommers
Copy link
Member

That's it for tonight for me.

@87
Copy link
Contributor Author
87 commented Feb 20, 2012

Okay, I'll take a look at it tomorrow.

@87
Copy link
Contributor Author
87 commented Feb 21, 2012

I tried to reproduce the issue with 64-bits localtime, but MinGW and win64 don't go together. I had to fix some issues with the installer, but after getting things to work on a 32-bit Python32 environment, it crashes after a few tests.

@87
Copy link
Contributor Author
87 commented Feb 21, 2012

Hmm, it also crashed on XP, using Python26 and using MinGW 4.6.1.

The crash happens for me in test_arrayprint, in the TestComplexArray class and can be reproduced by doing print np.array([0j], np.complex64)

@87
Copy link
Contributor Author
87 commented Feb 21, 2012

I pushed a few changes to add a function get_mktime to wrap mktime and _mktime64 and also added an improvement to the dll finder to look for the correct architecture.

@87
Copy link
Contributor Author
87 commented Feb 21, 2012

The test_datetime passes for me on WinXP with Python26 using MinGW. The test_multiarray crashes, however, just like the general test. (Is that the reason for you to stick to the older version of MinGW? I cannot find a download for it, though...)

@rgommers
Copy link
Member

This gives the same errors as before, on both Python 2.7 and 3.1:

======================================================================
ERROR: test_datetime_y2038 (test_datetime.TestDateTime)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "Z:\Users\rgommers\.wine\drive_c\Python27\lib\site-packages\numpy\core\tests\test_datetime.py", line 1707, in test_datetime_y2038
    assert_equal(str(a)[:-5], '2038-01-20T13:21:14')
OSError: Failed to use '_localtime64_s' to convert to a local time

======================================================================
ERROR: test_combinations (test_multiarray.TestArgmax)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "Z:\Users\rgommers\.wine\drive_c\Python27\lib\site-packages\numpy\core\tests\test_multiarray.py", line 979, in test_combinations
    assert_equal(np.argmax(arr), pos, err_msg="%r"%arr)
OSError: Failed to use '_localtime64_s' to convert to a local time

======================================================================
ERROR: test_combinations (test_multiarray.TestArgmin)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "Z:\Users\rgommers\.wine\drive_c\Python27\lib\site-packages\numpy\core\tests\test_multiarray.py", line 1047, in test_combinations
    assert_equal(np.argmin(arr), pos, err_msg="%r"%arr)
OSError: Failed to use '_localtime64_s' to convert to a local time

@87
Copy link
Contributor Author
87 commented Feb 21, 2012

Yeah, I figured. I'm a bit at a loss, though..

@mwiebe
Copy link
Member
mwiebe commented Feb 22, 2012

What I would suggest is instrumenting that part of the code so that before the call to the failing function call, it prints out the value of the parameters being passed in. This way, we could validate that nothing has munged the value before getting there, or create a separate tiny test program to experiment with what's going on.

@87
Copy link
Contributor Author
87 commented Feb 24, 2012

I cannot find the installer for MinGW 3.4.5, it is too old. I tried to download it from the Wayback Machine, but they don't have it either. The newest version of MinGW passes the datetime tests, but crashes on other tests.

Do you have a version I can download from somewhere, Ralf? Otherwise I'm afraid there is not much more I can do to solve these issues..
8000

@rgommers
Copy link
Member

The installer I used was MinGW-5.1.6.exe, which installs 3.4.5. All the part are still linked at http://wiki.codeblocks.org/index.php?title=MinGW_installation#GCC_3.4.5_manual_install for example. But I have to say that this is getting very questionable to use. Would be much better indeed to make gcc-4.x work. I'll have a look now if David's branch is public, that may solve your crashes.

@rgommers
Copy link
Member

His branch doesn't seem to be public, I asked on the list for it.

@87
Copy link
Contributor Author
87 commented Mar 1, 2012

Thanks! I've installed the separate files for version 3.4.5 from the given site, but I am running into some problems with it. At first MinGW could not find the .a files that were generated by the setup script (which I had to put into a MinGW directory manually), but now I get a few errors I cannot get around:

Running spawnve(0,'C:\\MinGW\\bin\\g++.exe',['C:\\MinGW\\bin\\g++.exe', '-mno-cygwin', '_configtest.o', '-lmsvcr90', '-o', '_configtest.exe'],os.environ)
_update_environment(...)
C:/MinGW/bin/../lib/gcc/mingw32/3.4.5/../../../../mingw32/lib/libmsvcr90.a(dkxds01142.o):(.text+0x0): multiple definition of `atexit'
C:/MinGW/bin/../lib/gcc/mingw32/3.4.5/../../../crt2.o:crt1.c:(.text+0x2b0): first defined here
C:/MinGW/bin/../lib/gcc/mingw32/3.4.5/../../../../mingw32/lib/libmsvcr90.a(dkxds00725.o):(.text+0x0): multiple definition of `_onexit'
C:/MinGW/bin/../lib/gcc/mingw32/3.4.5/../../../crt2.o:crt1.c:(.text+0x2c0): first defined here
collect2: ld returned 1 exit status
failure.

Which gives a "broken toolchain" error. Thing is, according to the symbol export, there should not be multiple definitions.
Did you run into these issues? (If so, how did you solve it?)

@rgommers
Copy link
Member
rgommers commented Mar 2, 2012

That is a very common issue, and looks familiar. Seems to be a MinGW bug, http://comments.gmane.org/gmane.comp.emulators.wine.devel/17437.

I documented how I set up Wine, see the "To install Wine on OS X..." paragraph at https://github.com/numpy/numpy/blob/master/doc/HOWTO_RELEASE.rst.txt. I think my mention of an issue with Wine + MinGW is exactly this, and I fixed it by installing Wine with http://code.google.com/p/osxwinebuilder/.

@charris
Copy link
Member
charris commented Mar 24, 2012

Where are we with this?

@rgommers
Copy link
Member

Down to 3 failures, but those are very hard to fix. MinGW is a real problem; 3.4.5 is very hard to even find, while 4.x is not supported. David was close to finishing the move to 4.x, but ran into some problems with DLLs and out of time. I'm not sure what the best way forward is at this point.

@charris
Copy link
Member
charris commented Apr 7, 2012 A3E2

Perhaps this could go in now and additional fixes could be made later. Does that seem reasonable?

@rgommers
Copy link
Member
rgommers commented Apr 8, 2012

That's fine with me.

@charris
Copy link
Member
charris commented Apr 15, 2012

Pushed in 913e55d..c17738a.

@charris charris closed this Apr 15, 2012
@thouis thouis mentioned this pull request Sep 27, 2012
Open
This was referenced Oct 16, 2012
Open
Closed
luyahan pushed a commit to plctlab/numpy that referenced this pull request Apr 25, 2024
@howjmay
Merge pull request numpy#214 from howjmay/vrsubhn
6dfa2ed 
feat: Add vrsubhn_[s16|s32|s64]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@87 @rgommers @mwiebe @charris
0