fix: deduplicate DT_NEEDED entries by SONAME#1022
fix: deduplicate DT_NEEDED entries by SONAME#1022deepakshirkem wants to merge 2 commits intoqualcomm:mainfrom
Conversation
When multiple shared libraries with the same SONAME are passed to the linker, ELD was emitting duplicate DT_NEEDED entries for the same SONAME.
parth-07
left a comment
There was a problem hiding this comment.
We need tests with this patch.
Additionally, we should warn if there are multiple different shared library with the same SO name.
lib/Target/GNULDBackend.cpp
Outdated
| const ELFDynObjectFile *dynObjFile = llvm::cast<ELFDynObjectFile>(lib); | ||
| if (addedLibs.count(dynObjFile->getInput()->getMemArea())) | ||
| std::string SOName = dynObjFile->getSOName(); | ||
| if (!addedSONAMEs.insert(SOName).second) |
There was a problem hiding this comment.
We are changing the if-condition without preserving the behavior of the old if-condition. Would it break anything?
There was a problem hiding this comment.
Hi @parth-07,
I tested this locally and the behavior is preserved. The old code deduplicated by MemoryArea pointer to prevent the same file from being added twice. The new code deduplicates by SONAME string,
which is strictly stronger:
So the new check covers everything the old one did, plus fixes the bug.
Screenshot With reproduce step at #50

There was a problem hiding this comment.
The old code deduplicated by MemoryArea pointer to prevent the same file from being added twice. The new code deduplicates by SONAME string,
which is strictly stronger:
What happens for libraries without any SONAME? Deduplication using SONAME wouldn't work for such libraries.
There was a problem hiding this comment.
Hi @parth-07, I investigated this concern and the fix handles this case correctly. Every SONAME library always has a SONAME set before our code runs
eld/lib/Readers/DynamicELFReader.cpp
Lines 122 to 123 in b03021e
getFallbackSOName() returns the resolved file path, which is unique per file. By the time our code runs in sizeDynamic(), every library already has a SONAME set either real or fallback. We never see an empty SONAME.
I am still open to your feedback!
|
Hi @parth-07, I will add a test in this patch. I also think emitting a warning for duplicate shared libraries with the same SONAME would make more sense. Thank You ((:: |
|
Hi @parth-07, Thank You ::)) |
| if (llvm::dyn_cast<ELFFileBase>(lib)->isELFNeeded()) { | ||
| const ELFDynObjectFile *dynObjFile = llvm::cast<ELFDynObjectFile>(lib); | ||
| if (addedLibs.count(dynObjFile->getInput()->getMemArea())) | ||
| std::string SOName = dynObjFile->getSOName(); |
There was a problem hiding this comment.
Hi @quic-seaswara, getSOName() returns std::string by value. Using llvm::StringRef here would result in a dangling reference since the temporary std::string would be destroyed at the end of the full expression.
I did try using llvm::StringRef initially but it resulted in a dangling reference error during compilation.
lib/Target/GNULDBackend.cpp
Outdated
| const ELFDynObjectFile *dynObjFile = llvm::cast<ELFDynObjectFile>(lib); | ||
| if (addedLibs.count(dynObjFile->getInput()->getMemArea())) | ||
| std::string SOName = dynObjFile->getSOName(); | ||
| std::string fileName = dynObjFile->getInput()->getFileName(); |
There was a problem hiding this comment.
can you move filenanme to 990 and use StringRef
lib/Target/GNULDBackend.cpp
Outdated
| llvm::StringSet<> addedSONAMEs; | ||
| llvm::StringMap<std::string> sonameToFile; | ||
| for (auto &lib : m_Module.getDynLibraryList()) { | ||
| if (llvm::dyn_cast<ELFFileBase>(lib)->isELFNeeded()) { |
There was a problem hiding this comment.
if (!isELFNeeded())
continue;
| RUN: %clang %clangopts -fPIC -c %p/Inputs/2.c -o %t1.2.o | ||
| RUN: %link %linkopts %t1.1.o -o %t1.lib1.so -shared -soname=foo | ||
| RUN: %link %linkopts %t1.2.o -o %t1.lib2.so -shared -soname=foo | ||
| RUN: %link %linkopts /dev/null %t1.lib1.so %t1.lib2.so -o %t1.out 2>&1 | %filecheck %s --check-prefix=WARN |
There was a problem hiding this comment.
why /dev/null ? is it portable to windows ?
| RUN: %link %linkopts %t1.2.o -o %t1.lib2.so -shared -soname=foo | ||
| RUN: %link %linkopts /dev/null %t1.lib1.so %t1.lib2.so -o %t1.out 2>&1 | %filecheck %s --check-prefix=WARN | ||
| RUN: %readelf --dynamic %t1.out | %filecheck %s --check-prefix=NEEDED | ||
| #WARN: Warning: Multiple shared libraries with the same SONAME "foo" detected |
There was a problem hiding this comment.
bfd and lld does not warn here, they simply emit the first shared library seen when there are multiple libraries with the same SONAME. I requested @deepakshirkem to emit the warning diagnostic here.
| "Empty archive file: '%0'") | ||
| DIAG(sframe_read_error, DiagnosticEngine::Error, | ||
| "SFrame Read Error : %0 from file %1") No newline at end of file | ||
| "SFrame Read Error : %0 from file %1") |
| "SFrame Read Error : %0 from file %1") No newline at end of file | ||
| "SFrame Read Error : %0 from file %1") | ||
| DIAG(warn_duplicate_soname, DiagnosticEngine::Warning, | ||
| "Multiple shared libraries with the same SONAME \"%0\" detected: %1 and %2") No newline at end of file |
There was a problem hiding this comment.
can you use the same format of message used by Gnu Ld or lld ?
| @@ -0,0 +1 @@ | |||
| int bar() { return 3; } No newline at end of file | |||
There was a problem hiding this comment.
you can have the same input file to generate two shared libraries ?
When multiple shared libraries with the same SONAME are passed to the linker, ELD was emitting duplicate DT_NEEDED entries. Signed-off-by: deepakshirkem <deepakshirke509@gmail.com>
1019252 to
5bd591c
Compare
|
Hi @quic-seaswara, I have addressed all the requested changes. Please take a look and let me know if any further changes are needed. Thank You ::)) |
Problem
Fixes #50
When multiple shared libraries with the same SONAME are passed to the linker, ELD emits duplicate DT_NEEDED entries for the same SONAME.
Fix
Deduplicate by SONAME string instead of file pointer using
llvm::StringSet<>. This matches the behavior ofld.lldandld.bfd.Testing
Verified manually using the reproducer from issue #50.