Problem with versioned functions (FreeBSD)

[kaj]

Trying to build rust_icu_ustring on FreeBSD (after getting around #235), I get a bunch of errors like this:

error[E0425]: cannot find function `u_strFromUTF8_70` in this scope
   --> /home/kaj/.cargo/registry/src/github.com-1ecc6299db9ec823/rust_icu_ustring-2.0.0/src/lib.rs:160:13
    |
160 |             versioned_function!(u_strFromUTF8)(
    |             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ help: a function with a similar name exists: `u_strFromUTF8`
    |
   ::: /usr/home/kaj/proj/r4s/target/debug/build/rust_icu_sys-f73623289ad55f94/out/lib.rs:3:850138
    |
3   | ..."] pub fn u_strFromUTF8 (dest : * mut UChar , destCapacity : i32 , pDestLength : * mut i32 , src : * const :: std :: os :: raw :: c_char , srcLength : i32 , pErrorCode : * mut UErrorCode) -> * mut UChar ; } ...
    |       --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- similarly named function `u_strFromUTF8` defined here
    |
    = note: this error originates in the macro `versioned_function` (in Nightly builds, run with -Z macro-backtrace for more info)

I get eleven such errors, all for different function names u_\w+_70. Checking the actual library, (with nm --dynamic --defined-only (libfile) it contains the symbols like u_strFromUTF8 but no symbols ending with _70 (or any other _number). The same command on linux (Fedora 35) shows u_strFromUTF8_69 (but no unversioned symbols).

The version number itself is correct, libicuuc.so on my FreeBSD host is a symlink to libicuuc.so.70.1. But the symbols in it are not versioned.

In rust_icu_sys/bindgen/macros.rs, it seems that versioning can be disabled by not using the renaming or the icu_version_in_env feature, but apparently at least one or those gets defined in my build even though it should not. Or should the cfg

rust_icu/rust_icu_sys/bindgen/macros.rs

Lines 32 to 39 in fc64773

	/// This macro will be used when no function renaming is needed.
	#[cfg(not(any(feature="renaming",feature="icu_version_in_env")))]
	#[macro_export]
	macro_rules! versioned_function {
	($func_name:path) => {
	$func_name
	}
	}

be changed to just

 #[cfg(not(feature="renaming"))] 

?

[kaj]

I believe I have up-to-date versions of the rust_icu_* crates:

: guran%; cargo tree -i rust_icu_sys
rust_icu_sys v2.0.0
├── rust_icu_common v2.0.0
│   ├── rust_icu_ucol v2.0.0
│   │   └── r4s v0.1.0 (/usr/home/kaj/proj/r4s)
│   ├── rust_icu_uenum v2.0.0
│   │   └── rust_icu_ucol v2.0.0 (*)
│   └── rust_icu_ustring v2.0.0
│       └── rust_icu_ucol v2.0.0 (*)
├── rust_icu_ucol v2.0.0 (*)
├── rust_icu_uenum v2.0.0 (*)
└── rust_icu_ustring v2.0.0 (*)

My rust toolchain is:

  stable-x86_64-unknown-freebsd unchanged - rustc 1.58.0 (02072b482 2022-01-11)

[filmil]

It is unexpected that only some of the symbols in the ICU library are renamed and some not.

be changed to just

#[cfg(not(feature="renaming"))]

That should not be correct, as

rust_icu/rust_icu_sys/bindgen/macros.rs

Line 9 in fc64773

#[cfg(all(feature="renaming",not(feature="icu_version_in_env")))]

is supposed to cover that feature combination.

[kaj]

@filmil huh? It seems to be line 9 and and 22 is for when renaming is enabled, so I think line 33 should cover both the cases when renaming is not enabled.

renaming	icu_version_in_env	selected
enabled	enabled	line 22
enabled	disabled	line 9
disabled	disabled	line 33
disabled	enabled	??

But maybe icu_version_in_env is never enabled unless renaming is? It seems to me that renaming is enabled when building on FreeBSD but should not be.

[kaj]

I also tried to disable features by changing my dependency to:

rust_icu_ucol = { version = "2.0.0", default-features = false }

but then I got the message:

error: You must use `renaming` when not using `use-bindgen`

So I changed the dependency to:

rust_icu_ucol = { version = "2.0.0", default-features = false, features = ["use-bindgen"] }

... and then I got this error:

error: couldn't read /usr/home/kaj/proj/r4s/target/debug/build/rust_icu_sys-00ce2788a020995a/out/macros.rs: No such file or directory (os error 2)
  --> /home/kaj/.cargo/registry/src/github.com-1ecc6299db9ec823/rust_icu_sys-2.0.0/src/lib.rs:45:1
   |
45 | include!(concat!(env!("OUT_DIR"), "/macros.rs"));
   | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
   |
   = note: this error originates in the macro `include` (in Nightly builds, run with -Z macro-backtrace for more info)

[kaj]

Some more info on this; checking out the rust_icu sources on my FreeBSD host, I find cargo test works fine in rust_icu_sys and rust_icu_common, but fails on versioned functions as above for other crates, such as rust_icu_ucol and rust_icu_uenum. So my present guess is that rust_icu_sys detects the unversioned symbol fine and works with them, but somehow fails to communicate this to downstream crates.

Thoughts on that? Any suggestions for things I can try?

[filmil]

On Sun, Jan 16, 2022 at 11:33 AM Rasmus Kaj ***@***.***> wrote: that rust_icu_sys detects the unversioned symbol fine and works with them, but somehow fails to communicate this to downstream crates. Thoughts on that? Any suggestions for things I can try?

This may well be. You must turn on the feature in *all* dependent crates to have it actually be used, even in transient dependencies. I am not quite sure why this is the case with cargo, but it seems to be. Message ID: ***@***.***>

…

[kaj]

It seems I was actually rather close above (#236 (comment)). If I edit the dependency like this it works:

rust_icu_ucol = { version = "2.0.0", default-features = false, features = ["use-bindgen", "icu_config"] }

... but of course, like that it does not build on Linux anymore. So I really think these flags should not be cargo features to be enabled or not from the consuming package at all, but instead autodetected flags that rust_icy_sys makes availiable for all dependent packages.

[filmil]

On Sun, Jan 16, 2022 at 3:19 PM Rasmus Kaj ***@***.***> wrote: ... but of course, like that it does not build on Linux anymore. So I really think these flags should not be cargo features to be enabled or not from the consuming package at all, but instead autodetected flags that rust_icy_sys makes availiable for all dependent packages.

rust_icu_sys does auto-detect the settings. But, the auto-detection is only as good as the pkg-config data that the system provides. If the pkg-config data is incorrect, and I have some suspicion that it might not be on your system based on the prior description of versioning, then one could expect issues. Since I don't have a FreeBSD to try this on, may I ask you to figure out the full configuration of ICU on your system and we could perhaps work from there. To wit, what are the `--cflags` in use for the various ICU library components? Thanks! Message ID: ***@***.***>

…

[yshui]

Same problem on Gentoo, icu-i18n.pc looks like this:

# Copyright (C) 2016 and later: Unicode, Inc. and others.
# License & terms of use: http://www.unicode.org/copyright.html
# Copyright (C) 2010-2013, International Business Machines Corporation. All Rights Reserved.

# CFLAGS contains only anything end users should set
CFLAGS =
# CXXFLAGS contains only anything end users should set
CXXFLAGS =
# DEFS only contains those UCONFIG_CPPFLAGS which are not auto-set by platform.h
DEFS =  -DU_DISABLE_RENAMING=1
prefix = /usr
exec_prefix = ${prefix}
#bindir = ${exec_prefix}/bin
libdir = /usr/lib
includedir = ${prefix}/include
baselibs = -lpthread -lm
#datarootdir = /usr/share
#datadir = /usr/share
#sbindir = ${exec_prefix}/sbin
#mandir = /usr/share/man
#sysconfdir = /etc
UNICODE_VERSION=14.0
ICUPREFIX=icu
ICULIBSUFFIX=
LIBICU=lib${ICUPREFIX}
#SHAREDLIBCFLAGS=-fPIC
pkglibdir=${libdir}/icu${ICULIBSUFFIX}/71.1
#pkgdatadir=${datadir}/icu${ICULIBSUFFIX}/71.1
ICUDATA_NAME = icudt71l
#ICUPKGDATA_DIR=/usr/lib
#ICUDATA_DIR=${pkgdatadir}
ICUDESC=International Components for Unicode

Version: 71.1
Cflags: -I${includedir}
# end of icu.pc.in
Description: International Components for Unicode: Internationalization library
Name: icu-i18n
Requires: icu-uc
Libs: -licui18n

I think icu_sys only checks --cflags?

[kaj]

Here's output from a bunch of pkg-config commands on my FreeBSD host:

:; pkg-config icu-i18n --cflags
-I/usr/local/include 
:; pkg-config icu-i18n --libs
-licui18n -L/usr/local/lib -licuuc -licudata 
:; pkg-config icu-i18n --variable=DEFS
-DU_DISABLE_RENAMING=1
:;
:; pkg-config icu-i18n --print-requires
icu-uc
:;
:; pkg-config icu-uc --cflags
-I/usr/local/include 
:; pkg-config icu-uc --libs
-L/usr/local/lib -licuuc -licudata 
:; pkg-config icu-uc --variable=DEFS
-DU_DISABLE_RENAMING=1

I guess the problem is that rust_icu doesn't find the -DU_DISABLE_RENAMING=1 flag, but I don't know if it is a bug in the FreeBSD pkgconfig data that it is not included in cflags or a bug in rust_icu that it doesn't look at --variable=DEFS.

The FreeBSD port of icu runs configure for icu with --disable-renaming (and some other flags) and does not seem to mess with the icu-*.pc files, so it seems that those are as intended by the icu project.

---

Looking some more, it seems that the omission of the U_DISABLE_RENAMING flag from --cflags is intentional. Instead, it is defined like this in unicode/uconfig.h:

/**
 * \def U_DISABLE_RENAMING
 * Determines whether to disable renaming or not.
 * @internal
 */
#ifndef U_DISABLE_RENAMING
#define U_DISABLE_RENAMING 1
#endif

So I guess the correct way of finding if renaming is enabled or not is to build and run a little program like this:

#include <unicode/uconfig.h>
#include <stdio.h>

int main() {
  if (U_DISABLE_RENAMING) {
    printf("Renaming disabled");
  } else {
    printf("Renaming enabled");
  }
}

... then compile that with proper flags from pkg-config, run it, and look at the output.