Re: Darwin 1.4 (OS X 10.1) Broken Compile, Snapshot and 7.1.3

Alex Avriette <a_avriette(at)acs(dot)org> writes:
> /usr/bin/ld: -undefined error must be used when -twolevel_namespace is in
> effect

All we know at this point is that Apple changed something between 10.0.4
and 10.1. If you want to look into it and figure out how to fix it,
that'd be great.

Attached is some possibly-relevant info that someone sent in recently.

regards, tom lane

> I recently asked (in pg-sql, because I did not know where to ask) for
> help with a problem 'making' pg on mac osx. I have finally tracked the
> problem down and have the appropriate information. It appears to be
> important to running on Mac OSX.
>
> Who can (should) I send it to? I am still trying to get pg to run under
> OSX 10.1.
>
> Ted
>
> -------------------------
>
> Mac OS X Developer Release Notes:
> Two-Level Namespace Executables
>
> ? The Problem (and the Solution)
> ? Using Two-Level Namespace Executables On Mac OS X 10.0
> ? Running Your Application as a Flat Namespace Executable
> ? Troubleshooting Two-Level Namespace Builds
> ? Building Your Application With PlugIns and Bundles
> ? New Dynamic Linking Functions for Use With Two-Level Namespace
> Executables
> ? NSAddImage
> ? NSLookupSymbolInImage
> ? NSIsSymbolNameDefinedInImage
>
> The Problem (and the Solution)
>
> The shared library implementation shipped with Mac OS X 10.0 is a
> robust, featureful implementation, with one small problem.
>
> When an executable file is loaded into a program, the dynamic linker
> (the part of the system responsible for loading code on demand)
> attempts to find all of the unresolved symbols (functions in other
> executable files). In order to do this, it needs to know which libraries
> contain those symbols. So when the program is built, you must
> specify the libraries that contain those functions to the static linker
> (the part of the build system responsible for linking object files
> together). The static linker places the names of those libraries into
> your program's executable file.
>
> The problem is that the static linker in Mac OS X 10.0 records only
> the names of the libraries, but not which functions are to be found in
> each of libraries. Therefore, it's not possible to load two libraries
> containing a definition of the same symbol name, because it's not
> possible to determine which library you really wanted the symbol
> from.
>
> For example, two libraries might both implement a "log" function.
> One of the libraries might be a math library, and the other might be a
> library for logging messages to the system console. If your program
> calls the math library "log," there is no way to guarantee that you are
> not calling the system console "log".
>
> This isn't usually a problem on Mac OS X 10.0, because the build
> system will give you a multiple-defined-symbols error message
> when you attempt to build your application using libraries that
> contain a symbol with the same name. But consider the following
> situations.
>
> ? Future versions of the system may export symbols which conflict
> with those implemented in your application. This will prevent users
> from being able to use your application.
> ? If your application supports third-party plugins or bundles, libraries
> used by your third-party developers may conflict with each other.
>
> To solve this problem, the Mac OS X 10.1 runtime environment
> supports a new feature that records the names of symbols expected
> to be found in a library along with the library name. An executable file
> built with this feature enabled is called a two-level namespace
> executable. An executable file without this feature is called a flat
> namespace executable.
>
> When your program links to a symbol defined in a subframework of
> an umbrella framework, the linker records the name of the umbrella
> framework as the parent library for the symbol. At runtime, the linker
> searches the umbrella framework, including all of the
> subframeworks it contains, to find the symbol.
>
> For example, if your program references the function MoveWindow,
> which is implemented in the HIToolbox subframework of the
> "Carbon" umbrella framework, "Carbon" will be recorded as the
> library name. Consequentially, if a later release of the Carbon
> umbrella framework moves the MoveWindow function to a new or
> different subframework (for example, a WindowManager framework),
> your program will continue to function.
>
> Using Two-Level Namespace Executables On Mac OS X 10.0
>
> The static linker enables the two-level namepace option
> (-twolevel_namespace) by default in Mac OS X 10.1. Two-level
> namespace executables are compatible with Mac OS X 10.0, where
> the linker will treat them as flat namespace executables (although
> your program may unexpectedly quit with a "multiple defined
> symbols" error if the resulting flat namespace contains multiple
> symbols with the same name).
>
> However, applications that you build as two-level namespace
> executables will cause versions of the update_prebinding tool up to
> and including the version installed with Mac OS X 10.0.4 to crash.
> update_prebinding is used by the Mac OS X installer to redo the
> prebinding optimization (see the Prebinding release note for more
> information). If the prebinding tool crashes, the installer will
> complete the installation, but the resulting system won't be fully
> prebound. To work around this problem, you have two options:
>
> ? You can use the -flat_namespace linker option to force the linker to
> build flat namespace executables. In Project Builder, add
> -flat_namespace to the OTHER_LDFLAGS Build Setting.
> ? Only system updates from Apple should cause update_prebinding
> to run. You can require Mac OS X 10.0.4, on the assumption that all
> prebinding has been completed with the final Mac OS X 10.0 system
> update. Note, however, that some third-party developers ship tools
> that do cause update_prebinding to run. Also, if Apple releases
> another 10.0 system update, which is unlikely but possible, this
> strategy may not work.
>
> Running Your Application as a Flat Namespace Executable
>
> If you want to test your two-level namespace as a flat namespace
> image on Mac OS X 10.1, set the
> DYLD_FORCE_FLAT_NAMESPACE environment variable to launch
> your program as a flat namespace executable.
>
> Note that when you do this, your program may terminate with a
> "multiple defined symbols" error if the resulting flat namespace
> contains multiple symbols with the same name.
>
> Troubleshooting Two-Level Namespace Builds
>
> Some existing projects will not link without modification, for the
> following reasons:
>
> ? There can be no unresolved, undefined references in two-level
> namespace executables. Using the -undefined suppress option will
> cause this error:
>
> /usr/bin/ld: -undefined error must be used when
> -twolevel_namespace is in effect
>
> If you recieve this error message, you need to remove the -undefined
> suppress option and make sure you specify the path to the program
> you are linking against with the option -bundle_loader
> pathnameToYourProgram. If you are linking against a framework,
> this path is already specified using the -framework option, so you
> just need to remove the -undefined suppress option. (To be really
> paranoid, specify the -undefined error option). See the next section
> for more information.
>
> ? When building a two-level namespace executable, you must link
> against all shared libraries containing the symbols you reference. If
> your program is currently a flat namespace executable, this may
> cause problems. When you build a flat namespace executable, the
> dynamic linker will search all libraries for the program's undefined
> symbols, even libraries that your code did not explicitly link against.
>
> For example, your application might link against
> ApplicationServices.framework, but not CoreFoundation.framework.
> Because ApplicationServices.framework links to
> CoreFoundation.framework, you can, as a flat namespace
> executable, use routines exported by CoreFoundation.framework. If
> you build the program as a two-level namespace executable, using
> routines exported by CoreFoundation.framework will result in
> undefined-symbol errors, so you must explicitly add
> CoreFoundation.framework to the list of libraries you link.
>
> If you use a symbol from a library that you are not explicitly linking
> against, you will get a single error message for each such library of
> the form:
>
> ld: object_file illegal reference to symbol: symbol
> defined in indirectly referenced dynamic library: library
>
> If you see this error message, you must add the library library to your
> link command.
>
> If library is a sub-framework of an umbrella (for example,
> HIToolbox.framework is a subframework of Carbon.framework), you
> will need to add the umbrella framework to your link objects (in
> Project Builder, just drag the framework into your project). Once you
> explicitly link against an umbrella framework, you may freely use all
> of the symbols it contains without referencing any sub-frameworks.
>
> Building Your Application With PlugIns and Bundles
>
> If your project includes bundles that must be resolved against
> symbols in the program that loads them, do not use the linker option
> -undefined suppress when building your bundles. Instead, use
> -undefined error and make sure you specify the path to the program
> you are linking against with the option -bundle_loader
> pathnameToYourProgramOrFramework. For Project Builder
> projects, add these arguments to the the OTHER_LDFLAGS build
> setting of the bundle's target (in the expert settings table of the Build
> Settings tab).
>
> Using the -bundle_loader option instead of -undefined suppress will
> cause an ordering problem if the bundles are located inside your
> application's bundle. For the application to copy the bundles to itself
> with a Copy Files build phase, the bundles must be built first.
> However, to properly link against the app and get their symbols
> resolved the bundles must build after the application.
>
> One solution for an application that loads plug-ins or other bundles
> is to provide your plug-in API implementation in a framework that
> bundles can link against. Your application will also link to that
> framework. For example, Interface Builder exposes an API for people
> who wish to write new palettes of objects. There is an
> InterfaceBuilder.framework that implements the functionality of the
> API. The Interface Builder application itself calls routines in this
> framework, and so do plug-in-palettes. This organization provides
> several benefits:
>
> ? It solves ordering issues and does not force plug-ins to link
> against your application's executable file.
> ? It provides good separation between routines available for plugins
> to use and routines that are internal to the app.
> ? It gives you a good place to package the plug-in shared library and
> all the headers and doc and so forth that developers will need to
> develop plug-ins. This makes it more convenient for your
> developers.
>
> Alternately, you can use a second application target called "Foo
> Wrapper" to fix the dependency problem:
>
> 1. Set the application name of the new target to the same name as
> the real application target ("Foo", in this example).
> 2. Set the original application target to not install itself.
> 3. Set the wrapper target to install itself.
> 4. Add a Copy Files phase to the wrapper target and remove all the
> other build phases.
> 5. Set up the Copy Files phase the same way as it was in the
> original app target.
> 6. Make the wrapper target depend on the app target and on all the
> bundle targets.
> 7. Make things that used to depend on the app target depend on the
> wrapper target instead.
> 8. If the app target was the first target in your target list, make the
> wrapper target first.
> 9. Finally, because the installation is now being performed by the
> Foo Wrapper target, but the actual application is being built by the
> Foo target, when the install happens, Project Builder will not remove
> debugging symbols from the application executable file. To fix this
> problem, add a shell script build phase to the Foo Wrapper target
> that is set to run only for install builds and that contains the
> following
> script:
>
> strip -S
> ${SYMROOT}/${PRODUCT_NAME}.${WRAPPER_EXTENSION}/Cont
> ents/MacOS/$ {PRODUCT_NAME}
>
> New Dynamic Linking Functions for Use With Two-Level
> Namespace Executables
>
> The runtime symbol lookup routines released in Mac OS X 10.0
> (located in the header <mach-o/dyld.h> and listed below) perform
> lookups in the flat, global symbol namespace, and thus, when you
> use them to find symbols in your plugins, may not return the
> intended symbols in for two-level namespace applications.
>
> The "hint" provided to NSLookupAndBindSymbolWithHint is a
> performance "hint" for the flat namespace lookup, and is thus not
> used as the first level name for a two-level namespace lookup. To
> perform a lookup within a two-level namespace executable, use the
> function NSLookupSymbolInImage, as documented below.
>
> Applications built as two-level namespace executables should
> instead use the following new routines.
>
> ? NSAddImage
> ? NSIsSymbolNameDefinedInImage
> ? NSLookupSymbolInImage
>
> NSAddImage
>
>
> const struct mach_header *
> NSAddImage(
> char *image_name,
> unsigned long options);
> #define NSADDIMAGE_OPTION_NONE 0x0
> #define NSADDIMAGE_OPTION_RETURN_ON_ERROR 0x1
> #define NSADDIMAGE_OPTION_WITH_SEARCHING 0x2
> #define NSADDIMAGE_OPTION_RETURN_ONLY_IF_LOADED 0x4
>
> NSAddImage loads the shared library specified by image_name into
> the current process, returning a pointer to the mach_header data
> structure of the loaded image. Any libraries that the specified library
> depends on are also loaded.
>
> If the shared library specified by image_name is already loaded, the
> mach_header already loaded is returned.
>
> The image_name parameter is a pointer to a C string containing the
> pathname to the shared library on disk. For best performance,
> specify the full pathname of the shared library?do not specify a
> symlink.
>
> The options parameter is a bit mask. Valid options are:
>
> ? NSADDIMAGE_OPTION_NONE
> No options.
>
> ? NSADDIMAGE_OPTION_RETURN_ON_ERROR
> If an error occurs and you have specified this option, NSAddImage
> returns NULL. You can then use the function NSLinkEditError to
> retrieve information about the error.
>
> If an error occurs, and you have not specified this option,
> NSAddImage will call the linkEdit error handler you have installed
> using the NSInstallLinkEditErrorHandlers function. If you have not
> installed a linkEdit error handler, NSAddImage prints an error to
> stderr and calls the exit function to end the program.
>
> ? NSADDIMAGE_OPTION_WITH_SEARCHING
> With this option the image_name passed for the library and all its
> dependents will be effected by the various DYLD environment
> variables as if this library were linked into the program.
>
> ? NSADDIMAGE_OPTION_RETURN_ONLY_IF_LOADED
> With this option, NSAddImage will not load a shared library that has
> not already been loaded. If the specified image_name is already
> loaded, NSAddImage will return its mach_header.
>
> The linkEdit error handler is documented in the NSModule(3) man
> page.
>
> NSLookupSymbolInImage
>
> extern NSSymbol
> NSLookupSymbolInImage(
> const struct mach_header *image,
> const char *symbolName
> unsigned long options);
> #define NSLOOKUPSYMBOLINIMAGE_OPTION_BIND 0x0
> #define NSLOOKUPSYMBOLINIMAGE_OPTION_BIND_NOW
> 0x1
> #define NSLOOKUPSYMBOLINIMAGE_OPTION_BIND_FULLY
> 0x2
> #define
> NSLOOKUPSYMBOLINIMAGE_OPTION_RETURN_ON_ERROR 0x4
>
> NSLookupSymbolInImage returns the specified symbol (as an
> NSSymbol) from the specified image.
>
> Error handling for NSLookupSymbolInImage is similar to error
> handling for NSAddImage.
>
> The image parameter is a pointer to a mach_header data structure.
> You can get this pointer from a shared library by calling
> NSAddImage.
>
> The symbolName parameter is a C string specifying the name of the
> symbol to retrieve.
>
> The options parameter is a bit mask. The following options are valid:
>
> ? NSLOOKUPSYMBOLINIMAGE_OPTION_BIND
> Bind the non-lazy symbols of the module in the image that defines
> the symbolName and let all lazy symbols in the module be bound on
> first call.
>
> This should be used in the case where you expect the module to
> bind without errors (for example, a library supplied with the system).
> If, later, you call a lazy symbol, and the lazy symbol fails to bind, the
> runtime will call the linkEdit handler you have installed using the
> NSInstallLinkEditErrorHandlers function. If there is no linkEdit
> handler installed, the runtime will print a message to stderr and call
> the exit function to end the program.
>
> ? NSLOOKUPSYMBOLINIMAGE_OPTION_BIND_NOW
> Bind all the non-lazy and lazy symbols of the module in the image
> that defines the symbolName and let all dependent symbols in the
> needed libraries be bound as needed. This would be used for a
> module that might not be expected bind without errors but links
> against only system-supplied libraries that are expected to bind
> without any errors. For example, you might use this option with a
> third-party plug-in.
>
> ? NSLOOKUPSYMBOLINIMAGE_OPTION_BIND_FULLY
> Bind all the symbols of the module that defines the symbolName
> and all if the dependent symbols of all needed libraries. This should
> only be used for things like signal handlers and linkEdit error
> handlers that can't bind other symbols once executed.
>
> ? NSLOOKUPSYMBOLINIMAGE_OPTION_RETURN_ON_ERROR
> Return NULL if the symbol cannot be bound. This option is similar to
> the NSAddImage option
> NSADDIMAGE_OPTION_RETURN_ON_ERROR. See NSAddImage
> for more details.
>
> NSIsSymbolNameDefinedInImage
>
> extern enum DYLD_BOOL
> NSIsSymbolNameDefinedInImage(
> const struct mach_header *image,
> const char *symbolName);
>
> NSIsSymbolNameDefinedInImage returns true if the specified
> image (or, if the image is a framework, any of its subframeworks)
> contains the specified symbol, false otherwise.
>
> The image parameter is a pointer to a mach_header data structure.
> You can get this pointer from a shared library by calling
> NSAddImage.
>
> The symbolName parameter is a C string specifying the name of the
> symbol to retrieve.
>
> The image parameter for NSLookupSymbolInImage and
> NSIsSymbolNameDefinedInImage is a pointer to the mach_header
> data structure of a Mach-O shared library. You can obtain a pointer to
> a mach_header data structure from:
>
> ? the NSAddImage function
> ? a linker defined symbol as defined in <mach-o/ldsym.h>,
> described on the ld(1) man page
> ? the dyld(3) function _dyld_get_image_header(3)
> ? the mach_header arguments to the callback functions called from
> _dyld_register_func_for_add_image(3).
>
> Copyright ? 2001 Apple Computer, Inc.