Skip to content

Commit 4355559

Browse files
iabyniabyn
committed
[MERGE] tidy typemap-finding code and fix priority
All but the last commit in this branch clean up, simplify, and better document and test the code which is responsible for searching for XS typemap files in standard locations (relative to both @inc and the current dir). They all (in theory) have no change in behaviour. The final commit restores the priority of files specified by -typemap: these were the highest priority before 5.10.0 and became lowest priority afterwards. Pre-5.10.0 was still the documented behaviour and I think the change was a mistake. In practice this last commit is unlikely to affect any real modules, but you never know.
2 parents 6e8e0e8 + b32a36b commit 4355559

File tree

7 files changed

+160
-105
lines changed

7 files changed

+160
-105
lines changed

‎dist/ExtUtils-ParseXS/lib/ExtUtils/ParseXS.pm‎

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -74,7 +74,6 @@ BEGIN {
7474
$VERSION = eval $VERSION if $VERSION =~ /_/;
7575

7676
use ExtUtils::ParseXS::Utilities qw(
77-
standard_typemap_locations
7877
trim_whitespace
7978
C_string
8079
valid_proto_string

‎dist/ExtUtils-ParseXS/lib/ExtUtils/ParseXS/Utilities.pm‎

Lines changed: 64 additions & 80 deletions
Original file line numberDiff line numberDiff line change
@@ -64,94 +64,75 @@ They are documented here for the benefit of future maintainers of this module.
6464
6565
=item * Purpose
6666
67-
Provide a list of filepaths where F<typemap> files may be found. The
68-
filepaths -- relative paths to files (not just directory paths) -- appear in this list in lowest-to-highest priority.
69-
70-
The highest priority is to look in the current directory.
71-
72-
'typemap'
73-
74-
The second and third highest priorities are to look in the parent of the
75-
current directory and a directory called F<lib/ExtUtils> underneath the parent
76-
directory.
77-
78-
'../typemap',
79-
'../lib/ExtUtils/typemap',
80-
81-
The fourth through ninth highest priorities are to look in the corresponding
82-
grandparent, great-grandparent and great-great-grandparent directories.
83-
84-
'../../typemap',
85-
'../../lib/ExtUtils/typemap',
86-
'../../../typemap',
87-
'../../../lib/ExtUtils/typemap',
88-
'../../../../typemap',
89-
'../../../../lib/ExtUtils/typemap',
90-
91-
The tenth and subsequent priorities are to look in directories named
92-
F<ExtUtils> which are subdirectories of directories found in C<@INC> --
93-
I<provided> a file named F<typemap> actually exists in such a directory.
94-
Example:
95-
96-
'/usr/local/lib/perl5/5.10.1/ExtUtils/typemap',
97-
98-
However, these filepaths appear in the list returned by
99-
C<standard_typemap_locations()> in reverse order, I<i.e.>, lowest-to-highest.
100-
101-
'/usr/local/lib/perl5/5.10.1/ExtUtils/typemap',
102-
'../../../../lib/ExtUtils/typemap',
103-
'../../../../typemap',
104-
'../../../lib/ExtUtils/typemap',
105-
'../../../typemap',
106-
'../../lib/ExtUtils/typemap',
107-
'../../typemap',
108-
'../lib/ExtUtils/typemap',
109-
'../typemap',
110-
'typemap'
67+
Returns a standard list of filepaths where F<typemap> files may be found.
68+
This will typically be something like:
69+
70+
map("$_/ExtUtils/typemap", reverse @INC),
71+
qw(
72+
../../../../lib/ExtUtils/typemap
73+
../../../../typemap
74+
../../../lib/ExtUtils/typemap
75+
../../../typemap
76+
../../lib/ExtUtils/typemap
77+
../../typemap
78+
../lib/ExtUtils/typemap
79+
../typemap
80+
typemap
81+
)
82+
83+
but the style of the pathnames may vary with OS. Note that the value to
84+
use for C<@INC> is passed as an array reference, and can be something
85+
other than C<@INC> itself.
86+
87+
Pathnames are returned in the order they are expected to be processed;
88+
this means that later files will update or override entries found in
89+
earlier files. So in particular, F<typemap> in the current directory has
90+
highest priority. C<@INC> is searched in reverse order so that earlier
91+
entries in C<@INC> are processed later and so have higher priority.
92+
93+
The values of C<-typemap> switches are not used here; they should be added
94+
by the caller to the list of pathnames returned by this function.
11195
11296
=item * Arguments
11397
114-
my @stl = standard_typemap_locations( \@INC );
98+
my @stl = standard_typemap_locations(\@INC);
11599
116-
Reference to C<@INC>.
100+
A single argument: a reference to an array to use as if it were C<@INC>.
117101
118102
=item * Return Value
119103
120-
Array holding list of directories to be searched for F<typemap> files.
104+
A list of F<typemap> pathnames.
121105
122106
=back
123107
124108
=cut
125109

126-
SCOPE: {
127-
my @tm_template;
110+
sub standard_typemap_locations {
111+
my $include_ref = shift;
128112

129-
sub standard_typemap_locations {
130-
my $include_ref = shift;
113+
my @tm;
131114

132-
if (not @tm_template) {
133-
@tm_template = qw(typemap);
134-
135-
my $updir = File::Spec->updir();
136-
foreach my $dir (
137-
File::Spec->catdir(($updir) x 1),
138-
File::Spec->catdir(($updir) x 2),
139-
File::Spec->catdir(($updir) x 3),
140-
File::Spec->catdir(($updir) x 4),
141-
) {
142-
unshift @tm_template, File::Spec->catfile($dir, 'typemap');
143-
unshift @tm_template, File::Spec->catfile($dir, lib => ExtUtils => 'typemap');
144-
}
145-
}
115+
# See function description above for why 'reverse' is used here.
116+
foreach my $dir (reverse @{$include_ref}) {
117+
my $file = File::Spec->catfile($dir, ExtUtils => 'typemap');
118+
push @tm, $file;
119+
}
146120

147-
my @tm = @tm_template;
148-
foreach my $dir (@{ $include_ref}) {
149-
my $file = File::Spec->catfile($dir, ExtUtils => 'typemap');
150-
unshift @tm, $file if -e $file;
151-
}
152-
return @tm;
121+
my $updir = File::Spec->updir();
122+
foreach my $dir (
123+
File::Spec->catdir(($updir) x 4),
124+
File::Spec->catdir(($updir) x 3),
125+
File::Spec->catdir(($updir) x 2),
126+
File::Spec->catdir(($updir) x 1),
127+
) {
128+
push @tm, File::Spec->catfile($dir, lib => ExtUtils => 'typemap');
129+
push @tm, File::Spec->catfile($dir, 'typemap');
153130
}
154-
} # end SCOPE
131+
132+
push @tm, 'typemap';
133+
134+
return @tm;
135+
}
155136

156137
=head2 C<trim_whitespace()>
157138
@@ -245,18 +226,21 @@ sub valid_proto_string {
245226
246227
=item * Purpose
247228
248-
Process all typemap files.
229+
Process all typemap files. Reads in any typemap files specified explicitly
230+
with C<-typemap> switches or similar, plus any typemap files found in
231+
standard locations relative to C<@INC> and the current directory.
249232
250233
=item * Arguments
251234
252235
my $typemaps_object = process_typemaps( $args{typemap}, $pwd );
253236
254-
List of two elements: C<typemap> element from C<%args>; current working
255-
directory.
237+
The first argument is the C<typemap> element from C<%args>; the second is
238+
the current working directory (which is only needed for error messages).
256239
257240
=item * Return Value
258241
259-
Upon success, returns an L<ExtUtils::Typemaps> object.
242+
Upon success, returns an L<ExtUtils::Typemaps> object which contains the
243+
accumulated results of all processed typemap files.
260244
261245
=back
262246
@@ -265,13 +249,13 @@ Upon success, returns an L<ExtUtils::Typemaps> object.
265249
sub process_typemaps {
266250
my ($tmap, $pwd) = @_;
267251

268-
my @tm = ref $tmap ? @{$tmap} : ($tmap);
252+
my @tm = standard_typemap_locations( \@INC );
269253

270-
foreach my $typemap (@tm) {
254+
my @explicit = ref $tmap ? @{$tmap} : ($tmap);
255+
foreach my $typemap (@explicit) {
271256
die "Can't find $typemap in $pwd\n" unless -r $typemap;
272257
}
273-
274-
push @tm, standard_typemap_locations( \@INC );
258+
push @tm, @explicit;
275259

276260
require ExtUtils::Typemaps;
277261
my $typemap = ExtUtils::Typemaps->new;

‎dist/ExtUtils-ParseXS/lib/ExtUtils/xsubpp‎

Lines changed: 23 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -67,18 +67,31 @@ B<xsubpp> [B<-v>] [B<-except>] [B<-s pattern>] [B<-prototypes>] [B<-noversionche
6767
This compiler is typically run by the makefiles created by L<ExtUtils::MakeMaker>
6868
or by L<Module::Build> or other Perl module build tools.
6969

70+
I<xsubpp> is just a thin wrapper over C<ExtUtils::ParseXS>.
71+
7072
I<xsubpp> will compile XS code into C code by embedding the constructs
7173
necessary to let C functions manipulate Perl values and creates the glue
7274
necessary to let Perl access those functions. The compiler uses typemaps to
73-
determine how to map C function parameters and variables to Perl values.
75+
determine how to map C function parameters and variables to Perl values
76+
and vice versa.
77+
78+
The compiler will search for typemap files called I<typemap>. It will use
79+
the following search path to find typemaps and apply them in that order,
80+
so later typemaps take precedence:
7481

75-
The compiler will search for typemap files called I<typemap>. It will use
76-
the following search path to find default typemaps, with the rightmost
77-
typemap taking precedence.
82+
map("$_/ExtUtils/typemap", reverse @INC),
7883

79-
../../../typemap:../../typemap:../typemap:typemap
84+
../../../../lib/ExtUtils/typemap
85+
../../../../typemap
86+
../../../lib/ExtUtils/typemap
87+
../../../typemap
88+
../../lib/ExtUtils/typemap
89+
../../typemap
90+
../lib/ExtUtils/typemap
91+
../typemap
92+
typemap
8093

81-
It will also use a default typemap installed as C<ExtUtils::typemap>.
94+
any files specifed by -typemap
8295

8396
=head1 OPTIONS
8497

@@ -97,9 +110,10 @@ Adds exception handling stubs to the C code.
97110

98111
=item B<-typemap typemap>
99112

100-
Indicates that a user-supplied typemap should take precedence over the
101-
default typemaps. This option may be used multiple times, with the last
102-
typemap having the highest precedence.
113+
Indicates that a user-supplied typemap should be applied in additon to any
114+
files found in the standard search path. This option may be used multiple
115+
times, with the last typemap having the highest precedence, and all such
116+
files processed after ones found in the standard path.
103117

104118
=item B<-output filename>
105119

‎dist/ExtUtils-ParseXS/t/101-standard_typemap_locations.t‎

Lines changed: 40 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
#!/usr/bin/perl
22
use strict;
33
use warnings;
4-
use Test::More tests => 3;
4+
use Test::More tests => 5;
55
use ExtUtils::ParseXS::Utilities qw(
66
standard_typemap_locations
77
);
@@ -17,21 +17,50 @@ use ExtUtils::ParseXS::Utilities qw(
1717
1
1818
unless @stl > 9;
1919

20-
# We check only as many location entries from the start of the array
21-
# (where the @INC-related entries are) as there are entries from @INC.
22-
# We manage to do that by stopping when we find the "updir" related
23-
# entries, which we assume is never contained in a default @INC entry.
24-
my $updir = File::Spec->updir;
20+
# Check that at least one typemap file can be found under @INC
2521
my $max = $#INC;
26-
$max = $#stl if $#stl < $max;
27-
foreach my $i (0.. $max) {
28-
$max = $i, last if $stl[$i] =~ /\Q$updir\E/;
29-
}
30-
3122
ok(
3223
( 0 < (grep -f $_, @stl[0..$max]) ),
3324
"At least one typemap file exists underneath \@INC directories"
3425
);
3526
}
3627
}
3728

29+
{
30+
my @fake_INC = qw(a/b/c d/e/f /g/h/i);
31+
my @expected =
32+
(
33+
map("$_/ExtUtils/typemap", reverse @fake_INC),
34+
qw(
35+
../../../../lib/ExtUtils/typemap
36+
../../../../typemap
37+
../../../lib/ExtUtils/typemap
38+
../../../typemap
39+
../../lib/ExtUtils/typemap
40+
../../typemap
41+
../lib/ExtUtils/typemap
42+
../typemap
43+
typemap
44+
)
45+
);
46+
47+
my @stl = standard_typemap_locations( \@fake_INC );
48+
49+
is(scalar @stl, scalar @expected,
50+
"with fake INC: corrrect number of entries in typemap locations list" );
51+
52+
SKIP: {
53+
# Only do a full string comparison on platforms which handle
54+
# "standard" pathname formats and '..' updirs. We *always* test
55+
# on linux, and otherwise test unless the second from last doesn't
56+
# look standard. Always testing on Linux means there is at least
57+
# one platform that won't falsely skip the test if garbage is
58+
# returned.
59+
skip "platform doesn't use ../..", 1
60+
if $^O ne 'linux'
61+
and $stl[-2] ne '../typemap';
62+
63+
is_deeply(\@stl, \@expected, "with fake INC: list of paths match");
64+
}
65+
}
66+

‎dist/ExtUtils-ParseXS/t/106-process_typemaps.t‎

Lines changed: 18 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,8 @@ use warnings;
44
use Carp;
55
use Cwd qw(cwd);
66
use File::Temp qw( tempdir );
7-
use Test::More tests => 2;
7+
use File::Spec;
8+
use Test::More tests => 6;
89
use ExtUtils::ParseXS::Utilities qw(
910
process_typemaps
1011
);
@@ -41,3 +42,19 @@ my $startdir = cwd();
4142
chdir $startdir;
4243
}
4344

45+
# Confirm that explicit typemaps via -typemap etc override standard
46+
# entries.
47+
48+
{
49+
my $tm_obj = process_typemaps(
50+
[ File::Spec->catfile("t", "data", "conflicting.typemap") ], '.');
51+
ok($tm_obj, "got typemap object");
52+
53+
my $tm_entry = $tm_obj->get_typemap(ctype => 'double');
54+
ok($tm_entry, "got typemap entry object");
55+
56+
my $xs = $tm_entry->xstype;
57+
ok($xs, "got typemap XS type");
58+
# should be overridden from T_NV
59+
is($xs, "T_DIFFERENT", "got typemap XS type");
60+
}

‎dist/ExtUtils-ParseXS/t/600-t-compat.t‎

Lines changed: 7 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -97,16 +97,20 @@ foreach my $test (@tests) {
9797
# typemap-parsing/handling code in ExtUtils::ParseXS looked like. For
9898
# bug-compatibility, we want to produce the same data structures as that
9999
# code as much as possible.
100+
# Except in 2025 the ordering was changed so that local files via -typemap
101+
# are now processed afterwards; the order has been changed here to reflect
102+
# that change.
100103
sub _process_typemaps {
101104
my ($tmap, $pwd) = @_;
102105

103-
my @tm = ref $tmap ? @{$tmap} : ($tmap);
106+
my @tm = standard_typemap_locations( \@INC );
104107

105-
foreach my $typemap (@tm) {
108+
my @explicit = ref $tmap ? @{$tmap} : ($tmap);
109+
foreach my $typemap (@explicit) {
106110
die "Can't find $typemap in $pwd\n" unless -r $typemap;
107111
}
112+
push @tm, @explicit;
108113

109-
push @tm, standard_typemap_locations( \@INC );
110114

111115
my ($type_kind_ref, $proto_letter_ref, $input_expr_ref, $output_expr_ref)
112116
= ( {}, {}, {}, {} );

‎pod/perldelta.pod‎

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -141,6 +141,14 @@ means it now requires a scalar argument, not a list, just like C<CORE::sleep>:
141141
# Evaluates @t in scalar context (giving the number of elements)
142142
# and sleeps for one second. It used to sleep for 42 seconds.
143143

144+
=item *
145+
146+
L<ExtUtils::ParseXS> has been upgraded from version 3.59 to 3.60. This
147+
release changes the ordering of the processing of typemap files which have
148+
been specified via C<xsubpp -typemap> arguments so that (as was the case
149+
prior to Perl 5.10.0) these files are applied last rather than first, and
150+
thus take priority over any system typemap files.
151+
144152
=back
145153

146154
=head2 Removed Modules and Pragmata

0 commit comments

Comments
 (0)