8000 BUG: f2py treats f90 module strings as variable names for dependent variables · Issue #28700 · numpy/numpy · GitHub
[go: up one dir, main page]
Skip to content

BUG: f2py treats f90 module strings as variable names for dependent variables #28700

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
m-weigand opened this issue Apr 14, 2025 · 0 comments
Open

BUG: f2py treats f90 module strings as variable names for dependent variables #28700

m-weigand opened this issue Apr 14, 2025 · 0 comments
Labels
00 - Bug

Comments

@m-weigand
Copy link
Contributor

Describe the issue:

While using f2py to generate a Python module from a legacy code base, I
encountered an error which I believe can be traced back to f2py not properly
recognizing strings when determining dependent variables.

The following minimal test case exhibits the encountered behaviour on numpy
2.3.0.dev0+git20241123.5f70dc8

! file: np_bug.f90
module mod1
	CHARACTER(6),PUBLIC,PARAMETER :: mkdir='mkdir '
	CHARACTER(7),PUBLIC,PARAMETER :: badvar2="badvar2"
end module mod1

module np_bug
	contains

	subroutine sub1
		use mod1
	end subroutine sub1
end module np_bug

Executing f2py as follows:

f2py --build-dir output \
    -m pot_f2py_bug \
    -c \
    --backend meson \
np_bug.f90

Triggers the following error:

Reading fortran codes...
		Reading file 'np_bug.f90' (format:free)
Post-processing...
		Block: pot_f2py_bug
						Block: mod1
						Block: np_bug
								Block: sub1
Traceback (most recent call last):
  File "/home/mweigand/.virtualenvs/numpy/bin/f2py", line 8, in <module>
	sys.exit(main())
			 ^^^^^^
  File "/home/mweigand/.virtualenvs/numpy/lib/python3.11/site-packages/numpy/f2py/f2py2e.py", line 781, in main
	run_compile()
  File "/home/mweigand/.virtualenvs/numpy/lib/python3.11/site-packages/numpy/f2py/f2py2e.py", line 727, in run_compile
	run_main(f" {' '.join(f2py_flags)} -m {modulename} {' '.join(sources)}".split())
  File "/home/mweigand/.virtualenvs/numpy/lib/python3.11/site-packages/numpy/f2py/f2py2e.py", line 473, in run_main
	postlist = callcrackfortran(files, options)
			   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/mweigand/.virtualenvs/numpy/lib/python3.11/site-packages/numpy/f2py/f2py2e.py", line 352, in callcrackfortran
	postlist = crackfortran.crackfortran(files)
			   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/mweigand/.virtualenvs/numpy/lib/python3.11/site-packages/numpy/f2py/crackfortran.py", line 3532, in crackfortran
	postlist = postcrack(grouplist[0])
			   ^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/mweigand/.virtualenvs/numpy/lib/python3.11/site-packages/numpy/f2py/crackfortran.py", line 2075, in postcrack
	g = postcrack(g, tab=tab + '\t')
		^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/mweigand/.virtualenvs/numpy/lib/python3.11/site-packages/numpy/f2py/crackfortran.py", line 2094, in postcrack
	block['body'] = analyzebody(block, args, tab=tab)
					^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/mweigand/.virtualenvs/numpy/lib/python3.11/site-packages/numpy/f2py/crackfortran.py", line 2266, in analyzebody
	b = postcrack(b, as_, tab=tab + '\t')
		^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/mweigand/.virtualenvs/numpy/lib/python3.11/site-packages/numpy/f2py/crackfortran.py", line 2094, in postcrack
	block['body'] = analyzebody(block, args, tab=tab)
					^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/mweigand/.virtualenvs/numpy/lib/python3.11/site-packages/numpy/f2py/crackfortran.py", line 2266, in analyzebody
	b = postcrack(b, as_, tab=tab + '\t')
		^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/mweigand/.virtualenvs/numpy/lib/python3.11/site-packages/numpy/f2py/crackfortran.py", line 2094, in postcrack
	block['body'] = analyzebody(block, args, tab=tab)
					^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/mweigand/.virtualenvs/numpy/lib/python3.11/site-packages/numpy/f2py/crackfortran.py", line 2266, in analyzebody
	b = postcrack(b, as_, tab=tab + '\t')
		^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/mweigand/.virtualenvs/numpy/lib/python3.11/site-packages/numpy/f2py/crackfortran.py", line 2090, in postcrack
	block['vars'] = analyzevars(block)
					^^^^^^^^^^^^^^^^^^
  File "/home/mweigand/.virtualenvs/numpy/lib/python3.11/site-packages/numpy/f2py/crackfortran.py", line 2636, in analyzevars
	params = get_parameters(vars, get_useparameters(block))
								  ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/mweigand/.virtualenvs/numpy/lib/python3.11/site-packages/numpy/f2py/crackfortran.py", line 2015, in get_useparameters
	params = get_parameters(mvars)
			 ^^^^^^^^^^^^^^^^^^^^^
  File "/home/mweigand/.virtualenvs/numpy/lib/python3.11/site-packages/numpy/f2py/crackfortran.py", line 2487, in get_parameters
	for n in get_sorted_names(vars):
			 ^^^^^^^^^^^^^^^^^^^^^^
  File "/home/mweigand/.virtualenvs/numpy/lib/python3.11/site-packages/numpy/f2py/crackfortran.py", line 2415, in get_sorted_names
	depend_dict = _calc_depend_dict(vars)
				  ^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/mweigand/.virtualenvs/numpy/lib/python3.11/site-packages/numpy/f2py/crackfortran.py", line 2410, in _calc_depend_dict
	_get_depend_dict(n, vars, depend_dict)
  File "/home/mweigand/.virtualenvs/numpy/lib/python3.11/site-packages/numpy/f2py/crackfortran.py", line 2396, in _get_depend_dict
	or _get_depend_dict(word, vars, deps):
	   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/mweigand/.virtualenvs/numpy/lib/python3.11/site-packages/numpy/f2py/crackfortran.py", line 2396, in _get_depend_dict
	or _get_depend_dict(word, vars, deps):
	   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/mweigand/.virtualenvs/numpy/lib/python3.11/site-packages/numpy/f2py/crackfortran.py", line 2396, in _get_depend_dict
	or _get_depend_dict(word, vars, deps):
	   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  [Previous line repeated 973 more times]
  File "/home/mweigand/.virtualenvs/numpy/lib/python3.11/site-packages/numpy/f2py/crackfortran.py", line 2388, in _get_depend_dict
	if '=' in vars[name] and not isstring(vars[name]):
								 ^^^^^^^^^^^^^^^^^^^^
  File "/home/mweigand/.virtualenvs/numpy/lib/python3.11/site-packages/numpy/f2py/auxfuncs.py", line 96, in isstring
	return isstring_or_stringarray(var) and not isarray(var)
		   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/mweigand/.virtualenvs/numpy/lib/python3.11/site-packages/numpy/f2py/auxfuncs.py", line 92, in isstring_or_stringarray
	return _ischaracter(var) and 'charselector' in var
		   ^^^^^^^^^^^^^^^^^
  File "/home/mweigand/.virtualenvs/numpy/lib/python3.11/site-packages/numpy/f2py/auxfuncs.py", line 71, in _ischaracter
	not isexternal(var)
		^^^^^^^^^^^^^^^
  File "/home/mweigand/.virtualenvs/numpy/lib/python3.11/site-packages/numpy/f2py/auxfuncs.py", line 412, in isexternal
	return 'attrspec' in var and 'external' in var['attrspec']
								 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
RecursionError: maximum recursion depth exceeded in comparison

Analysis of the problem

Running crackfortran.crack2fortran on the file shows the problem in the form of
the two Warnings that are printed:

On numpy 2.3.0.dev0+git20241123.5f70dc8:

In [4]: from numpy.f2py import crackfortran
   ...: mod = crackfortran.crackfortran('np_bug.f90')
   ...: crackfortran.crack2fortran(mod)
   ...:
Reading fortran codes...
        Reading file 'np_bug.f90' (format:free)
Post-processing...
        Block: mod1
        Block: np_bug
                Block: sub1                                                                                                                                                                   In: np_bug.f90:np_bug:sub1                                                                                                                                                                    get_useparameters: no module mod1 info used by sub1                                                                                                                                           Applying post-processing hooks...
  character_backward_compatibility_hook
Post-processing (stage 2)...
vars2fortran: Warning: cross-dependence between variables "mkdir" and "mkdir"
vars2fortran: Warning: cross-dependence between variables "badvar2" and "badvar2"

After applying the (temporary) fix detailed below:

In [1]: from numpy.f2py import crackfortran
   ...: mod = crackfortran.crackfortran('np_bug.f90')
   ...: crackfortran.crack2fortran(mod)
Reading fortran codes...
        Reading file 'np_bug.f90' (format:free)
Post-processing...
        Block: mod1
        Block: np_bug
                Block: sub1
In: np_bug.f90:np_bug:sub1
get_useparameters: no module mod1 info used by sub1
Applying post-processing hooks...
  character_backward_compatibility_hook
Post-processing (stage 2)...

I think the problem boils down to the regex in line 2625:

name_match = re.compile(r'[A-Za-z][\w$]*').match

(link to master:

numpy/numpy/f2py/crackfortran.py

Line 2625 in 67fb349

name_match = re.compile(r'[A-Za-z][\w$]*').match
)

The regex will also match if the variable name is included as a string (see
example above).
Unfortunately my regex skills are not a match for that challenge, so I just
resolved to removing any strings before checking for dependencies (patch
against 2.3.0.dev0, 5f70dc8):

diff --git a/numpy/f2py/crackfortran.py b/numpy/f2py/crackfortran.py
index 6eea034778..5ae058fcee 100644
--- a/numpy/f2py/crackfortran.py
+++ b/numpy/f2py/crackfortran.py
@@ -2911,7 +2911,18 @@ def compute_deps(v, deps):
				 vars[n]['depend'] = []
				 for v, m in list(dep_matches.items()):
					 if m(vars[n]['=']):
-                        vars[n]['depend'].append(v)
+                        # ignore strings
+                        cleaned_content = re.sub(
+                            r"('.*?')",
+                            '',
+                            re.sub(
+                                r'(".*?")',
+                                '',
+                                vars[n]['=']
+                            )
+                        )
+                        if m(cleaned_content):
+                            vars[n]['depend'].append(v)
				 if not vars[n]['depend']:
					 del vars[n]['depend']
			 if isscalar(vars[n]):

While I think this "fix" is a little bit too clumsy for a merge request, I
still confirmed that all old tests successfully run (spin test; spin test -m
slow).

Please let me know if I can help in any way with this issue.

Reproduce the code example:

Please see description above

Error message:



Python and NumPy Versions:








import sys, numpy; print(numpy.version); print(sys.version)

2.3.0.dev0+git20241123.5f70dc8

3.11.2 (main, Nov 30 2024, 21:22:50) [GCC 12.2.0]








Runtime Environment:


In [1]: import numpy; numpy.show_runtime()

[{'numpy_version': '2.3.0.dev0+git20241123.5f70dc8',

'python': '3.11.2 (main, Nov 30 2024, 21:22:50) [GCC 12.2.0]',

'uname': uname_result(system='Linux', node='titan', release='6.11.5+bpo-amd64', version='#1 SMP PREEMPT_DYNAMIC Debian 6.11.5-1~bpo12+1 (2024-11-11)', machine='x86_64')},

{'simd_extensions': {'baseline': ['SSE', 'SSE2', 'SSE3'],

'found': ['SSSE3',

'SSE41',

'POPCNT',

'SSE42',

'AVX',

'F16C',

'FMA3',

'AVX2'],

'not_found': ['AVX512F',

'AVX512CD',

'AVX512_KNL',

'AVX512_KNM',

'AVX512_SKX',

'AVX512_CLX',

'AVX512_CNL',

'AVX512_ICL',

'AVX512_SPR']}},

{'architecture': 'Haswell',

'filepath': '/usr/lib/x86_64-linux-gnu/openblas-pthread/libopenblasp-r0.3.21.so',

'internal_api': 'openblas',

'num_threads': 12,

'prefix': 'libopenblas',

'threading_layer': 'pthreads',

'user_api': 'blas',

'version': '0.3.21'}]


Context for the issue:


Fixing this would allow building python modules for our legacy fortran code.

      		
    

      
  





        



            
            

              
            

        

      


      
    








      


  




  
  

              

  
    Sign up for free
    to join this conversation on GitHub.
    Already have an account?
    Sign in to comment


  



  






  
                  




    

  


      
  

    Assignees
  



        

    No one assigned







      


  


  

    Labels
  



    

      

    00 - Bug









      

  

    

        

    Projects
  


        




    None yet




  



      


  

    
  

    Milestone
  


      No milestone





    
      
          


  

    

      
        

          
    

    Development
  




            
    
No branches or pull requests







      
    

  




      

    

  
      

  

    

      1 participant
    

    

        
          @m-weigand 
    

  






    

    




    






  

          

        



    



  





    

  


  


          



    



  

    

    

    





    


    

    

  



0