-
Notifications
You must be signed in to change notification settings - Fork 333
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add support for importing symbols from static library #902
base: main
Are you sure you want to change the base?
Changes from 3 commits
9b891d0
ede2555
ca0abbf
6a334a1
88114fc
77b6f89
e56a4c7
9d07856
e77b239
90043d4
cb9eb40
0ee2687
6f2a5ef
96f2766
f9ccb45
476099a
fcf38e6
a159f31
18b6aba
ecfe984
0db31a7
5e51921
405b85c
c68001b
ab90410
af07004
22ddfc2
f49b699
608b6b8
bc2023c
281a9d3
2e7d474
c9a59e5
37182a7
269cd6d
d42498d
0effd26
28e5ef7
9f53121
674ca5f
488dd58
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change | ||||
---|---|---|---|---|---|---|
|
@@ -79,6 +79,13 @@ The ExecutionEngine class | |||||
or a object file instance. Object file instance is not usable after this | ||||||
call. | ||||||
|
||||||
* .. method:: add_archive(archive_file) | ||||||
|
||||||
Add the symbols from the specified static archive file to the execution | ||||||
engine. It is a fatal error in LLVM if the *archive_file* does not exist. | ||||||
|
||||||
* *archive* str: a path to the static object file | ||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. The parameter name doesn't match that from above. Is "static object file" appropriate here, or is "archive file" more consistent?
Suggested change
|
||||||
|
||||||
* .. method:: set_object_cache(notify_func=None, getbuffer_func=None) | ||||||
|
||||||
Set the object cache callbacks for this engine. | ||||||
|
Original file line number | Diff line number | Diff line change | ||||||||
---|---|---|---|---|---|---|---|---|---|---|
|
@@ -10,7 +10,9 @@ | |||||||||
#include "llvm/Object/Binary.h" | ||||||||||
#include "llvm/Object/ObjectFile.h" | ||||||||||
#include "llvm/Support/Memory.h" | ||||||||||
|
||||||||||
#include "llvm/Support/MemoryBuffer.h" | ||||||||||
#include "llvm/Object/Archive.h" | ||||||||||
#include "llvm/Support/Errc.h" | ||||||||||
#include <cstdio> | ||||||||||
#include <memory> | ||||||||||
|
||||||||||
|
@@ -167,6 +169,40 @@ LLVMPY_MCJITAddObjectFile(LLVMExecutionEngineRef EE, LLVMObjectFileRef ObjF) { | |||||||||
{std::move(binary_tuple.first), std::move(binary_tuple.second)}); | ||||||||||
} | ||||||||||
|
||||||||||
API_EXPORT(int) | ||||||||||
LLVMPY_MCJITAddArchive(LLVMExecutionEngineRef EE, const char *ArchiveName, | ||||||||||
const char **OutError) { | ||||||||||
using namespace llvm; | ||||||||||
using namespace llvm::object; | ||||||||||
auto engine = unwrap(EE); | ||||||||||
|
||||||||||
ErrorOr<std::unique_ptr<MemoryBuffer>> ArBufOrErr = MemoryBuffer::getFile(ArchiveName); | ||||||||||
|
||||||||||
std::error_code EC = ArBufOrErr.getError(); | ||||||||||
if (EC){ | ||||||||||
*OutError = LLVMPY_CreateString(EC.message().c_str()); | ||||||||||
return 1; | ||||||||||
} | ||||||||||
|
||||||||||
Expected<std::unique_ptr<object::Archive>> ArchiveOrError = | ||||||||||
object::Archive::create(ArBufOrErr.get()->getMemBufferRef()); | ||||||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. The
Suggested change
|
||||||||||
|
||||||||||
if (!ArchiveOrError) { | ||||||||||
auto takeErr = ArchiveOrError.takeError(); | ||||||||||
std::string archiveErrStr = "Unable to load archive: " + std::string(ArchiveName); | ||||||||||
*OutError = LLVMPY_CreateString(archiveErrStr.c_str()); | ||||||||||
return 1; | ||||||||||
} | ||||||||||
|
||||||||||
std::unique_ptr<MemoryBuffer> &ArBuf = ArBufOrErr.get(); | ||||||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. These can also be simplified to:
|
||||||||||
std::unique_ptr<object::Archive> &Ar = ArchiveOrError.get(); | ||||||||||
object::OwningBinary<object::Archive> owningBinaryArchive(std::move(Ar), | ||||||||||
std::move(ArBuf)); | ||||||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
|
||||||||||
engine->addArchive(std::move(owningBinaryArchive)); | ||||||||||
*OutError = LLVMPY_CreateString(""); | ||||||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. We don't need to create an empty string if we're returning success.
Suggested change
|
||||||||||
return 0; | ||||||||||
} | ||||||||||
|
||||||||||
// | ||||||||||
// Object cache | ||||||||||
// | ||||||||||
|
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,5 @@ | ||
|
||
int __multiply_accumulate(int a, int b, int c) | ||
{ | ||
return (a * b) + c; | ||
} |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,4 @@ | ||
int __multiply_subtract(int a, int b, int c) | ||
{ | ||
return (a * b) - c; | ||
} |
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -16,6 +16,8 @@ | |
from llvmlite import binding as llvm | ||
from llvmlite.binding import ffi | ||
from llvmlite.tests import TestCase | ||
import llvmlite.tests | ||
from distutils.ccompiler import new_compiler | ||
|
||
|
||
# arvm7l needs extra ABI symbols to link successfully | ||
|
@@ -1935,6 +1937,54 @@ def test_get_section_content(self): | |
self.assertEqual(s.data().hex(), issue_632_text) | ||
|
||
|
||
class TestArchiveFile(BaseTest): | ||
mod_archive_asm = """ | ||
;ModuleID = <string> | ||
target triple = "{triple}" | ||
|
||
declare i32 @__multiply_accumulate(i32 %0, i32 %1, i32 %2) | ||
declare i32 @__multiply_subtract(i32 %0, i32 %1, i32 %2) | ||
""" | ||
|
||
@unittest.skipUnless(sys.platform.startswith('linux'), | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Why is this Linux specific? It needs to be tested on all platforms. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. @apmasell thanks I modified the code to test on all platforms. I am now getting errors on all versions of Windows and a single version of macOS. Do you have any feedback you can offer? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. One thing that stands out to me is that the library is called There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Perhaps using There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. @gmarkall thanks that got me further on Windows, still more Windows errors to debug. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. On Windows you can't delete an open file. I wonder if the file is still open when you attempt to clean up the temp dir, causing the issue you see - perhaps deleting |
||
"Linux-specific test") | ||
def test_add_archive(self): | ||
target_machine = self.target_machine(jit=False) | ||
|
||
jit = llvm.create_mcjit_compiler(self.module(self.mod_archive_asm), | ||
target_machine) | ||
|
||
# Create compiler with default options | ||
c = new_compiler() | ||
workdir = os.path.dirname(llvmlite.tests.__file__) + "/" | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. It can't be assumed that the llvmlite test dir is writable (and I don't think it's good form to drop files into a package dir as part of running the tests anyway). If the test dir isn't writable then the test errors:
|
||
|
||
# Compile into .o files | ||
file1 = workdir + "a.c" | ||
file2 = workdir + "b.c" | ||
|
||
objects = c.compile([file1, file2]) | ||
library_name = "foo" | ||
# Create static or shared library | ||
|
||
c.create_static_lib(objects, library_name, output_dir=workdir) | ||
|
||
static_library_name = workdir + "lib" + library_name + ".a" | ||
jit.add_archive(static_library_name) | ||
|
||
mac_func = CFUNCTYPE(c_int, c_int, c_int, c_int)( | ||
jit.get_function_address("__multiply_accumulate")) | ||
|
||
msub_func = CFUNCTYPE(c_int, c_int, c_int, c_int)( | ||
jit.get_function_address("__multiply_subtract")) | ||
|
||
self.assertEqual(mac_func(10, 10, 20), 120) | ||
self.assertEqual(msub_func(10, 10, 20), 80) | ||
|
||
os.unlink(objects[0]) | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. If the test fails then this leaves files on disk. The |
||
os.unlink(objects[1]) | ||
os.unlink(static_library_name) | ||
|
||
|
||
class TestTimePasses(BaseTest): | ||
def test_reporting(self): | ||
mp = llvm.create_module_pass_manager() | ||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What does the word "static" mean here? My understanding is that in general, an archive contains unlinked objects, and those objects might eventually take part in a static or dynamic linking process (depending on how they were compiled, of course).
According to the LLVM docs, it looks like adding an archive uses
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@gmarkall static here means static archive (i.e libc.a) vs shared library (libc.so). I double checked the code on the LLVM C++ side of things and it is only expecting a static archive.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It looks like somehow my comment got truncated, apologies - what I'm trying to suggest is that "static archive" feels like a weird term - it's an archive file of objects, which will probably be linked statically. So either "archive file" or "static library" seem like appropriate terms (but "archive file" is better here since the method name refers to archives), but "static archive" is a mashup of the two, which I can't find any reference to in common use.