Fix import error on aarch64: define 'long' using ctypes.c_long

[vsevolod-misiul]

Summary

On Linux ARM64 (aarch64) platforms, installing PyOpenGL_accelerate fails with the following error:

Error compiling Cython file:
...
    self.buffer = long( buffers )
                          ^
src/vbo.pyx:191:26: undeclared name not builtin: long

This happens because the built-in long type is not defined in Python 3 on ARM64 platforms, unlike some legacy Python 2 or x86-based assumptions. This breaks Cython compilation when trying to cast buffer identifiers.

This patch imports long from ctypes.c_long, which resolves the issue on these platforms.

Changes

• Added from ctypes import c_long as long to handle missing long type on aarch64.

Platform tested

• Linux on ARM64 (Ubuntu 22.04, aarch64)

[ChrisBarker-NOAA]

I think this is only an issue with Cython 3 -- but we should be using Cython 3 by now!

Does this seem to work with Cython3 across the board?

NOTE: I'm working on a conda-forge recipe -- so I may apply this as a patch to build .... (though pinning to Cython<3 may be the more expedient route...)

[ChrisBarker-NOAA]

Got it all working on conda-forge -- but I had to add this patch -- so it would be nice to get this merged.

I do wonder how the wheels got built ????

[vsevolod-misiul]

I think this is only an issue with Cython 3 -- but we should be using Cython 3 by now!

Does this seem to work with Cython3 across the board?

NOTE: I'm working on a conda-forge recipe -- so I may apply this as a patch to build .... (though pinning to Cython<3 may be the more expedient route...)

Yes, this fix should work across the board with Cython 3. Using ctypes.c_long is the standard way to get a platform-appropriate C long type, so it should be safe on x86_64, and other platforms, preventing similar issues there as well.

[ChrisBarker-NOAA]

Indeed -- and it is working for me -- with this patch and Cython 3 -- and with Cython <3, I got segfaults.

I think -- numpy 2 changed a bit how it handles the C long types, which might explain the segfaults. I've had other issues with that).

I curse C for defining "short" and "long" way back when, rather than the modern int32 and int64, etc ... oh well.

Anyway, no need to understand all the details -- what I think we need isthis patch and also to pin Cython to >3

Maybe add that pin to this PR?

Also: I see in the MANIFEST.in file that the generated c files are still being included in the source distro -- that probably should change.

[vsevolod-misiul]

Indeed -- and it is working for me -- with this patch and Cython 3 -- and with Cython <3, I got segfaults.

I think -- numpy 2 changed a bit how it handles the C long types, which might explain the segfaults. I've had other issues with that).

I curse C for defining "short" and "long" way back when, rather than the modern int32 and int64, etc ... oh well.

Anyway, no need to understand all the details -- what I think we need isthis patch and also to pin Cython to >3

Maybe add that pin to this PR?

Also: I see in the MANIFEST.in file that the generated c files are still being included in the source distro -- that probably should change.

As you requested, I've added the version pin to require cython>=3.0.0 in pyproject.toml and just pushed the change. This should ensure robust builds going forward.

[ChrisBarker-NOAA]

Thanks! now we just need a merge ...

[vsevolod-misiul]

Thanks! now we just need a merge ...

I've been looking forward to this for a month. 😄

[ChrisBarker-NOAA]

@mcfletch: This is actually pretty bad -- pip install of accelerate on OS-X Intel fails for this reason.

There is no wheel for OS-X Intel -- so ti tries to build from source.

Also -- I'm curious how the wheels that are there were successfully built, and if they actually work (I could build with Cython < 3, but got segfaults when trying to run the tests).

BTW -- I'm sure there's a couple of us that would be glad to help with maintenance of you don't have the time.

[ChrisBarker-NOAA]

NOTE:

I've got a build for conda-forge working:

https://anaconda.org/conda-forge/pyopengl-accelerate

But I needed to apply this patch to get it to build -- see:

https://github.com/conda-forge/pyopengl-accelerate-feedstock

[swt2c]

While this might fix the compile issue, I'm suspicious that it might not be a correct fix. I'm seeing these test failures after applying the patch.

=================================== FAILURES ===================================
________________ TestCoreDatatype.test_glbuffersubdata_numeric _________________

>   ???
E   TypeError: an integer is required

src/vbo.pyx:192: TypeError

During handling of the above exception, another exception occurred:

self = <test_arraydatatype.TestCoreDatatype testMethod=test_glbuffersubdata_numeric>

    @pytest.mark.skipif(not np, reason="Numpy not available")
    def test_glbuffersubdata_numeric(self):
        from OpenGL.arrays import vbo
    
        assert vbo.get_implementation()
        points = np.array(
            [
                [0, 0, 0],
                [0, 1, 0],
                [1, 0.5, 0],
                [1, 0, 0],
                [1.5, 0.5, 0],
                [1.5, 0, 0],
            ],
            dtype='f',
        )
        d = vbo.VBO(points)
>       with d:

pyopengl-3.1.9/tests/test_arraydatatype.py:150: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
src/vbo.pyx:236: in OpenGL_accelerate.vbo.VBO.bind
    ???
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

>   ???
E   TypeError: 'int' object is not subscriptable

src/vbo.pyx:194: TypeError
______________________________ TestCore.test_vbo _______________________________

>   ???
E   TypeError: an integer is required

src/vbo.pyx:192: TypeError

During handling of the above exception, another exception occurred:

self = <test_core.TestCore testMethod=test_vbo>

    def test_vbo(self):
        """Test utility vbo wrapper"""
        from OpenGL.arrays import vbo
    
        if not vbo.get_implementation():
            return
        if not glVertexPointerd or not glDrawElements:
            return
        points = array(
            [
                [0, 0, 0],
                [0, 1, 0],
                [1, 0.5, 0],
                [1, 0, 0],
                [1.5, 0.5, 0],
                [1.5, 0, 0],
            ],
            dtype='d',
        )
        indices = array(
            range(len(points)),
            ['i', 'I'][int(bool(OpenGL.ERROR_ON_COPY))],  # test coercion if we can
        )
        d = vbo.VBO(points)
        glDisable(GL_CULL_FACE)
        glNormal3f(0, 0, 1)
        glColor3f(1, 1, 1)
        glEnableClientState(GL_VERTEX_ARRAY)
        try:
            for x in range(1, 255, 10):
>               d.bind()

pyopengl-3.1.9/tests/test_core.py:229: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
src/vbo.pyx:236: in OpenGL_accelerate.vbo.VBO.bind
    ???
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

>   ???
E   TypeError: 'int' object is not subscriptable

src/vbo.pyx:194: TypeError
_______________________________ test_sf_2980896 ________________________________

>   ???
E   TypeError: an integer is required

src/vbo.pyx:192: TypeError

During handling of the above exception, another exception occurred:

    @pytest.mark.skipif(not psutil, reason="No psutil available")
    @pytest.mark.skipif(not np, reason="No Numpy available")
    @pygamegltest.pygametest()
    def test_sf_2980896():
        """Test SF#2980896 report of memory leak on VBO transfer"""
        data = arrays.GLfloatArray.zeros((1000,))
        memory = get_current_memory()
        for i in range(8):
            new_vbo = vbo.VBO(data)
>           with new_vbo:

pyopengl-3.1.9/tests/test_vbo_memusage.py:39: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
src/vbo.pyx:236: in OpenGL_accelerate.vbo.VBO.bind
    ???
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

>   ???
E   TypeError: 'int' object is not subscriptable

src/vbo.pyx:194: TypeError
=============================== warnings summary ===============================
../../../../../usr/lib64/python3.14/site-packages/pygame/pkgdata.py:25
  /usr/lib64/python3.14/site-packages/pygame/pkgdata.py:25: DeprecationWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html
    from pkg_resources import resource_stream, resource_exists

../../../../../usr/lib64/python3.14/site-packages/numpy/_pytesttester.py:79
  /usr/lib64/python3.14/site-packages/numpy/_pytesttester.py:79: PytestCollectionWarning: cannot collect 'test' because it is not a function.
    def __call__(self, label='fast', verbose=1, extra_argv=None,

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
=========================== short test summary info ============================
FAILED pyopengl-3.1.9/tests/test_arraydatatype.py::TestCoreDatatype::test_glbuffersubdata_numeric
FAILED pyopengl-3.1.9/tests/test_core.py::TestCore::test_vbo - TypeError: 'in...
FAILED pyopengl-3.1.9/tests/test_vbo_memusage.py::test_sf_2980896 - TypeError...
= 3 failed, 85 passed, 2 skipped, 3 deselected, 1 xfailed, 2 warnings in 15.67s =

[ChrisBarker-NOAA]

@swt2c:

I see you're using python 3.14 -- maybe somethting has changed there.

What's the version of everything else?

• Python
• Cython
• numpy
• platfrom?
• architecture?

[totaam]

You should try using int instead of long:
#147 (comment)

[ChrisBarker-NOAA]

Well, it should match what the OpenGL headers have.

On some platforms, int and long are the same, and others not -- one of those really annoying things about C!

Also, rather than the ctypes long or int, we should use cython.long or cython.int.

https://cython.readthedocs.io/en/latest/src/userguide/language_basics.html#types

[ChrisBarker-NOAA]

One more note:

This patch worked, and passed the tests, on all the platforms supported by conda-forge.

https://github.com/conda-forge/pyopengl-accelerate-feedstock

Including windows, where int and long are the same, and Linux and Mac where they are not.

So I think long is correct.

[swt2c]

One more note:

This patch worked, and passed the tests, on all the platforms supported by conda-forge.

https://github.com/conda-forge/pyopengl-accelerate-feedstock

Including windows, where int and long are the same, and Linux and Mac where they are not.

So I think long is correct.

But the code in question is Python code, not C code. So I think @totaam is correct that int is what should be used. I have not been able to test yet.

EDIT: answered my own question. It looks like pyopengl-accelerate-feedstock is running part of the test suite, but not all of it.

============================= test session starts =============================
platform win32 -- Python 3.12.11, pytest-8.4.1, pluggy-1.6.0
rootdir: %SRC_DIR%
collected 53 items

tests\test_arraydatatypeaccel.py .................................       [ 62%]
tests\test_numpyaccel.py ....................                            [100%]

============================= 53 passed in 0.40s ==============================

[totaam]

@ChrisBarker-NOAA I would have to look at the generated C code to be sure, but doesn't using int(..) create a python integer object? This is not a cast, and Python ints don't have any clipping issues.

[ChrisBarker-NOAA]

I don't have time to dig in , but in Cython (particularly used in this way) you are setting the type to match the C type, so it doesn't have to do Python<->C conversions -- so you really want the C type that matches the C type used in the OpenGL code you are calling.

Key is that this DOES work with the whole set of platforms the conda-forge builds for:

It runs the opengl-accelerate tests -- it doesn't run the PyOpenGL tests, as those require OpenGL on the test runners. maybe those could be run on Windows and/or the Mac -- though I think that I had trouble getting them to run locally, so give up trying.

Actually -- what we should have is tests on the gitHub CI -- which I started, but there's the whole headless problem:

#152

But back to this issue:

IIUC, the "new" problem was on python 3.14 and I don't know what numpy versiuon or platform -- This error may have nothing to do with this patch.

The error is "TypeError: 'int' object is not subscriptable"

C int, C long, python int -- none of them are subscriptable.

I think the problem is here:

    def create_buffers( self ):
        """Create the internal buffer(s)"""
        assert not self.created, """Already created the buffer"""
        buffers = self.get_implementation().glGenBuffers(1)
        try:
            self.buffer = long( buffers )
        except (TypeError,ValueError) as err:
            self.buffer = buffers[0]

which means that self.buffer = long( buffers ) is raising a TypeError or ValueError and then it's trying to index it (maybe assuming it's an array?

buffers isn't typed, so it's whatever:

buffers = self.get_implementation().glGenBuffers(1)

is returning ...

this is all quite confusing, as what I see above (line 39)

cdef public unsigned int buffer

so buffer is typed as a (C) unsigned int.

back to the code at hand:

        try:
            self.buffer = long( buffers )
        except (TypeError,ValueError) as err:
            self.buffer = buffers[0]

that may well have been a conversion to a Python2 long (required because a Python2 int couldn't hold the full range of a C unsigned int) -- and then Cython is auto-converting it to a C unsigned int.

If that's the case, then you'd want to use self.buffer = int( buffers )

As suggested here -- as a Python3 int is a Python2 long.

I wonder what type buffers is ??? Why isn't that known at this point???

Anyway -- lots of testing to be done ...

Antoher hint:

assert isinstance( other, (int,long) ), """Only know how to add integer/long offsets"""

so that must have been a Python2 long

[ChrisBarker-NOAA]

I really don't have the time for this, but since I was thinking about it....

I tested just using int and all seems well -- at least on my Mac with python3.9 -- I'll see about updating the conda packages, which will give us a range of tests ...

new PR here:

#155

@vsevolod-misiul: -- I'm pretty sure this is the right way to go -- so if you could test my PR, that would be great -- and then you can close this one.

[swt2c]

Just confirmed on my end as well, switching to int fixes the issue with the tests and also seems to be the correct solution.

I submitted a PR as well with just the long to int fix: #156

[ChrisBarker-NOAA]

Yet another note:

Turns out that the tests are not run on the conda packages on Linux:

  commands:
    - pytest tests  # [not linux]

that's because they failed, probably due to missing OPenGL on the headless CI servers -- so maybe it's been broken on Linux all along.