Description
So I have a 1.07 GB json file that I am trying to open. However, the kernel crashes every time I try to run these three lines:
import pandas as pd
f = open("large json file", "r")
pd.read_json(f)
Anyone have any ideas on this? Is it a defect?
Python 2.7.3
[GCC 4.7.2] on linux2
*** glibc detected *** python: double free or corruption (out): 0x00007fff0d86a480 ***
======= Backtrace: =========
/lib/x86_64-linux-gnu/libc.so.6(+0x76d76)[0x7fd1b46b2d76]
/lib/x86_64-linux-gnu/libc.so.6(cfree+0x6c)[0x7fd1b46b7aac]
/usr/local/lib/python2.7/dist-packages/pandas/json.so(JSON_DecodeObject+0x149)[0x7fd1afc35909]
/usr/local/lib/python2.7/dist-packages/pandas/json.so(JSONToObj+0x248)[0x7fd1afc33078]
python(PyEval_EvalFrameEx+0x69f0)[0x4b2ce0]
python(PyEval_EvalFrameEx+0xaf0)[0x4acde0]
======= Memory map: ========
00400000-00658000 r-xp 00000000 08:03 2269915 /usr/bin/python2.7
00857000-00858000 r--p 00257000 08:03 2269915 /usr/bin/python2.7
00858000-008c1000 rw-p 00258000 08:03 2269915 /usr/bin/python2.7
008c1000-008d3000 rw-p 00000000 00:00 0
02075000-036ca000 rw-p 00000000 00:00 0 [heap]
7fd164000000-7fd164021000 rw-p 00000000 00:00 0
7fd164021000-7fd168000000 ---p 00000000 00:00 0
7fd1699e3000-7fd1aeaf2000 rw-p 00000000 00:00 0
7fd1aeaf2000-7fd1aeafb000 r-xp 00000000 08:03 2370198 /usr/local/lib/python2.7/dist-packages/pandas/_testing.so
7fd1aeafb000-7fd1aecfa000 ---p 00009000 08:03 2370198 /usr/local/lib/python2.7/dist-packages/pandas/_testing.so
7fd1aecfa000-7fd1aecfb000 r--p 00008000 08:03 2370198 /usr/local/lib/python2.7/dist-packages/pandas/_testing.so
7fd1aecfb000-7fd1aecfc000 rw-p 00009000 08:03 2370198 /usr/local/lib/python2.7/dist-packages/pandas/_testing.so
7fd1aecfc000-7fd1aeffd000 rw-p 00000000 00:00 0
7fd1aeffd000-7fd1af0e5000 r-xp 00000000 08:03 2327743 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.17
7fd1af0e5000-7fd1af2e5000 ---p 000e8000 08:03 2327743 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.17
7fd1af2e5000-7fd1af2ed000 r--p 000e8000 08:03 2327743 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.17
7fd1af2ed000-7fd1af2ef000 rw-p 000f0000 08:03 2327743 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.17
7fd1af2ef000-7fd1af304000 rw-p 00000000 00:00 0
7fd1af304000-7fd1af31a000 r-xp 00000000 08:03 2370192 /usr/local/lib/python2.7/dist-packages/pandas/msgpack.so
7fd1af31a000-7fd1af51a000 ---p 00016000 08:03 2370192 /usr/local/lib/python2.7/dist-packages/pandas/msgpack.so
7fd1af51a000-7fd1af51b000 r--p 00016000 08:03 2370192 /usr/local/lib/python2.7/dist-packages/pandas/msgpack.so
7fd1af51b000-7fd1af51e000 rw-p 00017000 08:03 2370192 /usr/local/lib/python2.7/dist-packages/pandas/msgpack.so
7fd1af51e000-7fd1af5e0000 rw-p 00000000 00:00 0
7fd1af5e0000-7fd1af5ef000 r-xp 00000000 08:03 2319476 /usr/lib/python2.7/lib-dynload/pyexpat.so
7fd1af5ef000-7fd1af7ee000 ---p 0000f000 08:03 2319476 /usr/lib/python2.7/lib-dynload/pyexpat.so
7fd1af7ee000-7fd1af7ef000 r--p 0000e000 08:03 2319476 /usr/lib/python2.7/lib-dynload/pyexpat.so
7fd1af7ef000-7fd1af7f1000 rw-p 0000f000 08:03 2319476 /usr/lib/python2.7/lib-dynload/pyexpat.so
7fd1af7f1000-7fd1af818000 r-xp 00000000 08:03 5276611 /lib/x86_64-linux-gnu/libexpat.so.1.6.0
7fd1af818000-7fd1afa18000 ---p 00027000 08:03 5276611 /lib/x86_64-linux-gnu/libexpat.so.1.6.0
7fd1afa18000-7fd1afa1a000 r--p 00027000 08:03 5276611 /lib/x86_64-linux-gnu/libexpat.so.1.6.0
7fd1afa1a000-7fd1afa1b000 rw-p 00029000 08:03 5276611 /lib/x86_64-linux-gnu/libexpat.so.1.6.0
7fd1afa1b000-7fd1afa28000 r-xp 00000000 08:03 2319452 /usr/lib/python2.7/lib-dynload/_elementtree.so
7fd1afa28000-7fd1afc27000 ---p 0000d000 08:03 2319452 /usr/lib/python2.7/lib-dynload/_elementtree.so
7fd1afc27000-7fd1afc28000 r--p 0000c000 08:03 2319452 /usr/lib/python2.7/lib-dynload/_elementtree.so
7fd1afc28000-7fd1afc29000 rw-p 0000d000 08:03 2319452 /usr/lib/python2.7/lib-dynload/_elementtree.so
7fd1afc29000-7fd1afc3d000 r-xp 00000000 08:03 2370203 /usr/local/lib/python2.7/dist-packages/pandas/json.so
7fd1afc3d000-7fd1afe3d000 ---p 00014000 08:03 2370203 /usr/local/lib/python2.7/dist-packages/pandas/json.so
7fd1afe3d000-7fd1afe3e000 r--p 00014000 08:03 2370203 /usr/local/lib/python2.7/dist-packages/pandas/json.so
7fd1afe3e000-7fd1afe3f000 rw-p 00015000 08:03 2370203 /usr/local/lib/python2.7/dist-packages/pandas/json.so
7fd1afe3f000-7fd1afe8a000 r-xp 00000000 08:03 2370199 /usr/local/lib/python2.7/dist-packages/pandas/parser.so
7fd1afe8a000-7fd1b0089000 ---p 0004b000 08:03 2370199 /usr/local/lib/python2.7/dist-packages/pandas/parser.so
7fd1b0089000-7fd1b008a000 r--p 0004a000 08:03 2370199 /usr/local/lib/python2.7/dist-packages/pandas/parser.so
7fd1b008a000-7fd1b008f000 rw-p 0004b000 08:03 2370199 /usr/local/lib/python2.7/dist-packages/pandas/parser.so
7fd1b008f000-7fd1b0090000 rw-p 00000000 00:00 0
7fd1b0090000-7fd1b00cb000 r-xp 00000000 08:03 2370195 /usr/local/lib/python2.7/dist-packages/pandas/_sparse.so
7fd1b00cb000-7fd1b02ca000 ---p 0003b00 08:03 2370195 /usr/local/lib/python2.7/dist-packages/pandas/_sparse.so
7fd1b02ca000-7fd1b02cb000 r--p 0003a000 08:03 2370195 /usr/local/lib/python2.7/dist-packages/pandas/_sparse.so
7fd1b02cb000-7fd1b02cf000 rw-p 0003b00 08:03 2370195 /usr/local/lib/python2.7/dist-packages/pandas/_sparse.so
7fd1b02cf000-7fd1b02d0000 rw-p 00000000 00:00 0
7fd1b02d0000-7fd1b02fb000 r-xp 00000000 08:03 2370191 /usr/local/lib/python2.7/dist-packages/pandas/index.so
7fd1b02fb000-7fd1b04fb000 ---p 0002b000 08:03 2370191 /usr/local/lib/python2.7/dist-packages/pandas/index.so
7fd1b04fb000-7fd1b04fc000 r--p 0002b000 08:03 2370191 /usr/local/lib/python2.7/dist-packages/pandas/index.so
7fd1b04fc000-7fd1b04ff000 rw-p 0002c000 08:03 2370191 /usr/local/lib/python2.7/dist-packages/pandas/index.so
7fd1b04ff000-7fd1b0500000 rw-p 00000000 00:00 0
7fd1b0500000-7fd1b06e6000 r-xp 00000000 08:03 2370193 /usr/local/lib/python2.7/dist-packages/pandas/algos.so
7fd1b06e6000-7fd1b08e5000 ---p 001e6000 08:03 2370193 /usr/local/lib/python2.7/dist-packages/pandas/algos.so
7fd1b08e5000-7fd1b08e6000 r--p 001e5000 08:03 2370193 /usr/local/lib/python2.7/dist-packages/pandas/algos.so
7fd1b08e6000-7fd1b08f9000 rw-p 001e6000 08:03 2370193 /usr/local/lib/python2.7/dist-packages/pandas/algos.so
7fd1b08f9000-7fd1b08fc000 rw-p 00000000 00:00 0
7fd1b08fc000-7fd1b0902000 r-xp 00000000 08:03 2319447 /usr/lib/python2.7/lib-dynload/_csv.so
7fd1b0902000-7fd1b0b01000 ---p 00006000 08:03 2319447 /usr/lib/python2.7/lib-dynload/_csv.so
7fd1b0b01000-7fd1b0b02000 r--p 00005000 08:03 2319447 /usr/lib/python2.7/lib-dynload/_csv.so
7fd1b0b02000-7fd1b0b04000 rw-p 00006000 08:03 2319447 /usr/lib/python2.7/lib-dynload/_csv.so
7fd1b0b04000-7fd1b0bc0000 r-xp 00000000 08:03 2370196 /usr/local/lib/python2.7/dist-packages/pandas/lib.so
7fd1b0bc0000-7fd1b0dc0000 ---p 000bc000 08:03 2370196 /usr/local/lib/python2.7/dist-packages/pandas/lib.so
7fd1b0dc0000-7fd1b0dc1000 r--p 000bc000 08:03 2370196 /usr/local/lib/python2.7/dist-packages/pandas/lib.so
7fd1b0dc1000-7fd1b0dce000 rw-p 000bd000 08:03 2370196 /usr/local/lib/python2.7/dist-packages/pandas/lib.so
7fd1b0dce000-7fd1b0dd2000 rw-p 00000000 00:00 0
7fd1b0dd2000-7fd1b0dd5000 r-xp 00000000 08:03 2319453 /usr/lib/python2.7/lib-dynload/_hashlib.so
7fd1b0dd5000-7fd1b0fd5000 ---p 00003000 08:03 2319453 /usr/lib/python2.7/lib-dynload/_hashlib.so
7fd1b0fd5000-7fd1b0fd6000 r--p 00003000 08:03 2319453 /usr/lib/python2.7/lib-dynload/_hashlib.so
7fd1b0fd6000-7fd1b0fd7000 rw-p 00004000 08:03 2319453 /usr/lib/python2.7/lib-dynload/_hashlib.so
7fd1b0fd7000-7fd1b118e000 r-xp 00000000 08:03 2327265 /usr/lib/x86_64-linux-gnu/libcrypto.so.1.0.0
7fd1b118e000-7fd1b138d000 ---p 001b7000 08:03 2327265 /usr/lib/x86_64-linux-gnu/libcrypto.so.1.0.0
7fd1b138d000-7fd1b13a8000 r--p 001b6000 08:03 2327265 /usr/lib/x86_64-linux-gnu/libcrypto.so.1.0.0
7fd1b13a8000-7fd1b13b7000 rw-p 001d1000 08:03 2327265 /usr/lib/x86_64-linux-gnu/libcrypto.so.1.0.0
7fd1b13b7000-7fd1b13bb000 rw-p 00000000 00:00 0
7fd1b13bb000-7fd1b1411000 r-xp 00000000 08:03 2327740 /usr/lib/x86_64-linux-gnu/libssl.so.1.0.0
7fd1b1411000-7fd1b1610000 ---p 00056000 08:03 2327740 /usr/lib/x86_64-linux-gnu/libssl.so.1.0.0
7fd1b1610000-7fd1b1613000 r--p 00055000 08:03 2327740 /usr/lib/x86_64-linux-gnu/libssl.so.1.0.0
7fd1b1613000-7fd1b161a000 rw-p 00058000 08:03 2327740 /usr/lib/x86_64-linux-gnu/libssl.so.1.0.0
7fd1b161a000-7fd1b1622000 r-xp 00000000 08:03 2319461 /usr/lib/python2.7/lib-dynload/_ssl.so
7fd1b1622000-7fd1b1821000 ---p 00008000 08:03 2319461 /usr/lib/python2.7/lib-dynload/_ssl.so
7fd1b1821000-7fd1b1822000 r--p 00007000 08:03 2319461 /usr/lib/python2.7/lib-dynload/_ssl.so
7fd1b1822000-7fd1b1823000 rw-p 00008000 08:03 2319461 /usr/lib/python2.7/lib-dynload/_ssl.so
7fd1b1823000-7fd1b18f1000 r-xp 00000000 08:03 2370202 /usr/local/lib/python2.7/dist-packages/pandas/tslib.so
7fd1b18f1000-7fd1b1af0000 ---p 000ce000 08:03 2370202 /usr/local/lib/python2.7/dist-packages/pandas/tslib.so
7fd1b1af0000-7fd1b1af1000 r--p 000cd000 08:03 2370202 /usr/local/lib/python2.7/dist-packages/pandas/tslib.so
7fd1b1af1000-7fd1b1aff000 rw-p 000ce000 08:03 2370202 /usr/local/lib/python2.7/dist-packages/pandas/tslib.so
7fd1b1aff000-7fd1b1b02000 rw-p 00000000 00:00 0
7fd1b1b02000-7fd1b1b28000 r-xp 00000000 08:03 2319448 /usr/lib/python2.7/lib-dynload/_ctypes.so
7fd1b1b28000-7fd1b1d27000 ---p 00026000 08:03 2319448 /usr/lib/python2.7/lib-dynload/_ctypes.so
7fd1b1d27000-7fd1b1d28000 r--p 00025000 08:03 2319448 /usr/lib/python2.7/lib-dynload/_ctypes.so
7fd1b1d28000-7fd1b1d2c000 rw-p 00026000 08:03 2319448 /usr/lib/python2.7/lib-dynload/_ctypes.so
7fd1b1d2c000-7fd1b1d2d000 rw-p 00000000 00:00 0
7fd1b1d2d000-7fd1b1d75000 r-xp 00000000 08:03 1024002 /usr/local/lib/python2.7/dist-packages/numpy/random/mtrand.so
7fd1b1d75000-7fd1b1f75000 ---p 00048000 08:03 1024002 /usr/local/lib/python2.7/dist-packages/numpy/random/mtrand.so
7fd1b1f75000-7fd1b1f76000 r--p 00048000 08:03 1024002 /usr/local/lib/python2.7/dist-packages/numpy/random/mtrand.so
7fd1b1f76000-7fd1b1fac000 rw-p 00049000 08:03 1024002 /usr/local/lib/python2.7/dist-packages/numpy/random/mtrand.so
7fd1b1fac000-7fd1b1fad000 rw-p 00000000 00:00 0
7fd1b1fad000-7fd1b1fb6000 r-xp 00000000 08:03 1040387 /usr/local/lib/python2.7/dist-packages/numpy/fft/fftpack_lite.so
7fd1b1fb6000-7fd1b21b5000 ---p 00009000 08:03 1040387 /usr/local/lib/python2.7/dist-packages/numpy/fft/fftpack_lite.so
7fd1b21b5000-7fd1b21b6000 r--p 00008000 08:03 1040387 /usr/local/lib/python2.7/dist-packages/numpy/fft/fftpack_lite.so
7fd1b21b6000-7fd1b21b7000 rw-p 00009000 08:03 1040387 /usr/local/lib/python2.7/dist-packages/numpy/fft/fftpack_lite.so
7fd1b21b7000-7fd1b21b8000 r-xp 00000000 08:03 2319470 /usr/lib/python2.7/lib-dynload/future_builtins.so
7fd1b21b8000-7fd1b23b7000 ---p 00001000 08:03 2319470 /usr/lib/python2.7/lib-dynload/future_builtins.so
7fd1b23b7000-7fd1b23b8000 r--p 00000000 08:03 2319470 /usr/lib/python2.7/lib-dynload/future_builtins.so
7fd1b23b8000-7fd1b23b9000 rw-p 00001000 08:03 2319470 /usr/lib/python2.7/lib-dynload/future_builtins.so
7fd1b23b9000-7fd1b23d8000 r-xp 00000000 08:03 2319455 /usr/lib/python2.7/lib-dynload/_io.so
7fd1b23d8000-7fd1b25d7000 ---p 0001f000 08:03 2319455 /usr/lib/python2.7/lib-dynload/_io.so
7fd1b25d7000-7fd1b25d8000 r--p 0001e000 08:03 2319455 /usr/lib/python2.7/lib-dynload/_io.so
7fd1b25d8000-7fd1b25e1000 rw-p 0001f000 08:03 2319455 /usr/lib/python2.7/lib-dynload/_io.so
7fd1b25e1000-7fd1b275e000 r-xp 00000000 08:03 1392651 /usr/local/lib/python2.7/dist-packages/numpy/linalg/_umath_linalg.so
7fd1b275e000-7fd1b295e000 ---p 0017d000 08:03 1392651 /usr/local/lib/python2.7/dist-packages/numpy/linalg/_umath_linalg.so
7fd1b295e000-7fd1b295f000 r--p 0017d000 08:03 1392651 /usr/local/lib/python2.7/dist-packages/numpy/linalg/_umath_linalg.so
7fd1b295f000-7fd1b2960000 rw-p 0017e000 08:03 1392651 /usr/local/lib/python2.7/dist-packages/numpy/linalg/_umath_linalg.so
7fd1b2960000-7fd1b29fc000 rw-p 00000000 00:00 0
7fd1b29fc000-7fd1b2b61000 r-xp 00000000 08:03 999427 /usr/local/lib/python2.7/dist-packages/numpy/linalg/lapack_lite.so
7fd1b2b61000-7fd1b2d60000 ---p 00165000 08:03 999427 /usr/local/lib/python2.7/dist-packages/numpy/linalg/lapack_lite.so
7fd1b2d60000-7fd1b2d61000 r--p 00164000 08:03 999427 /usr/local/lib/python2.7/dist-packages/numpy/linalg/lapack_lite.so
7fd1b2d61000-7fd1b2d62000 rw-p 00165000 08:03 999427 /usr/local/lib/python2.7/dist-packages/numpy/linalg/lapack_lite.so
7fd1b2d62000-7fd1b2dfd000 rw-p 00000000 00:00 0
7fd1b2dfd000-7fd1b2e03000 r-xp 00000000 08:03 1122311 /usr/local/lib/python2.7/dist-packages/numpy/lib/_compiled_base.so
7fd1b2e03000-7fd1b3002000 ---p 00006000 08:03 1122311 /usr/local/lib/python2.7/dist-packages/numpy/lib/_compiled_base.soAborted