Nokogiri 1.8.4 installation challenges on macOS Mojave

[deepj]

If you're having trouble installing Nokogiri ...

Have you tried following [the installation tutorial][tutorial]?
yes

What is the output of gem install?

Building native extensions with: '--use-system-libraries --with-xslt-dir=/usr/local/opt/libxslt --with-xslt-dir=/usr/local/opt/libxml2'
This could take a while...
ERROR:  Error installing nokogiri:
	ERROR: Failed to build gem native extension.

    current directory: ~/.gem/ruby/2.5.1/gems/nokogiri-1.8.4/ext/nokogiri
~/.rubies/ruby-2.5.1/bin/ruby -r ./siteconf20180926-34637-1628dkt.rb extconf.rb --use-system-libraries --with-xslt-dir=/usr/local/opt/libxslt --with-xslt-dir=/usr/local/opt/libxml2
checking if the C compiler accepts ... yes
checking if the C compiler accepts -Wno-error=unused-command-line-argument-hard-error-in-future... no
Building nokogiri using system libraries.
ERROR: cannot discover where libxml2 is located on your system. please make sure `pkg-config` is installed.
*** extconf.rb failed ***
Could not create Makefile due to some reason, probably lack of necessary
libraries and/or headers.  Check the mkmf.log file for more details.  You may
need configuration options.

Provided configuration options:
	--with-opt-dir
	--without-opt-dir
	--with-opt-include
	--without-opt-include=${opt-dir}/include
	--with-opt-lib
	--without-opt-lib=${opt-dir}/lib
	--with-make-prog
	--without-make-prog
	--srcdir=.
	--curdir
	--ruby=~/.rubies/ruby-2.5.1/bin/$(RUBY_BASE_NAME)
	--help
	--clean
	--use-system-libraries=true
	--with-zlib-dir
	--without-zlib-dir
	--with-zlib-include
	--without-zlib-include=${zlib-dir}/include
	--with-zlib-lib
	--without-zlib-lib=${zlib-dir}/lib
	--with-xml2-dir
	--without-xml2-dir
	--with-xml2-include
	--without-xml2-include=${xml2-dir}/include
	--with-xml2-lib
	--without-xml2-lib=${xml2-dir}/lib
	--with-libxml-2.0-config
	--without-libxml-2.0-config
	--with-pkg-config
	--without-pkg-config
	--with-xslt-dir
	--with-xslt-include
	--without-xslt-include=${xslt-dir}/include
	--with-xslt-lib
	--without-xslt-lib=${xslt-dir}/lib
	--with-exslt-dir
	--without-exslt-dir
	--with-exslt-include
	--without-exslt-include=${exslt-dir}/include
	--with-exslt-lib
	--without-exslt-lib=${exslt-dir}/lib
	--with-libexslt-config
	--without-libexslt-config

To see why this extension failed to compile, please check the mkmf.log which can be found here:

  ~/.gem/ruby/2.5.1/extensions/x86_64-darwin-18/2.5.0-static/nokogiri-1.8.4/mkmf.log

extconf failed, exit code 1

Gem files will remain installed in ~/.gem/ruby/2.5.1/gems/nokogiri-1.8.4 for inspection.
Results logged to ~/.gem/ruby/2.5.1/extensions/x86_64-darwin-18/2.5.0-static/nokogiri-1.8.4/gem_make.out

What are the contents of the mkmf.log file?

"clang -o conftest -I~/.rubies/ruby-2.5.1/include/ruby-2.5.0/x86_64-darwin18 -I~/.rubies/ruby-2.5.1/include/ruby-2.5.0/ruby/backward -I~/.rubies/ruby-2.5.1/include/ruby-2.5.0 -I. -I~/.rubies/ruby-2.5.1/include  -D_XOPEN_SOURCE -D_DARWIN_C_SOURCE -D_DARWIN_UNLIMITED_SELECT -D_REENTRANT    -O3 -Wno-error=shorten-64-to-32  -pipe  conftest.c  -L. -L~/.rubies/ruby-2.5.1/lib -L. -L~/.rubies/ruby-2.5.1/lib  -fstack-protector -L/usr/local/lib     -lruby.2.5.1-static -framework Foundation  -lpthread -lgmp -ldl -lobjc  "
checked program was:
/* begin */
1: #include "ruby.h"
2:
3: int main(int argc, char **argv)
4: {
5:   return 0;
6: }
/* end */

"clang -I~/.rubies/ruby-2.5.1/include/ruby-2.5.0/x86_64-darwin18 -I~/.rubies/ruby-2.5.1/include/ruby-2.5.0/ruby/backward -I~/.rubies/ruby-2.5.1/include/ruby-2.5.0 -I. -I~/.rubies/ruby-2.5.1/include  -D_XOPEN_SOURCE -D_DARWIN_C_SOURCE -D_DARWIN_UNLIMITED_SELECT -D_REENTRANT    -O3 -Wno-error=shorten-64-to-32  -pipe    -Werror -c conftest.c"
checked program was:
/* begin */
1: #include "ruby.h"
2:
3: int main() {return 0;}
/* end */

"clang -I~/.rubies/ruby-2.5.1/include/ruby-2.5.0/x86_64-darwin18 -I~/.rubies/ruby-2.5.1/include/ruby-2.5.0/ruby/backward -I~/.rubies/ruby-2.5.1/include/ruby-2.5.0 -I. -I~/.rubies/ruby-2.5.1/include  -D_XOPEN_SOURCE -D_DARWIN_C_SOURCE -D_DARWIN_UNLIMITED_SELECT -D_REENTRANT    -O3 -Wno-error=shorten-64-to-32  -pipe  -Wno-error=unused-command-line-argument-hard-error-in-future   -Werror -c conftest.c"
error: unknown warning option '-Werror=unused-command-line-argument-hard-error-in-future'; did you mean '-Werror=unused-command-line-argument'? [-Werror,-Wunknown-warning-option]
checked program was:
/* begin */
1: #include "ruby.h"
2:
3: int main() {return 0;}
/* end */

"pkg-config --exists libxml-2.0"
| pkg-config --libs libxml-2.0
=> "-lxml2\n"
"clang -o conftest -I~/.rubies/ruby-2.5.1/include/ruby-2.5.0/x86_64-darwin18 -I~/.rubies/ruby-2.5.1/include/ruby-2.5.0/ruby/backward -I~/.rubies/ruby-2.5.1/include/ruby-2.5.0 -I. -I~/.rubies/ruby-2.5.1/include  -D_XOPEN_SOURCE -D_DARWIN_C_SOURCE -D_DARWIN_UNLIMITED_SELECT -D_REENTRANT    -O3 -Wno-error=shorten-64-to-32  -pipe  conftest.c  -L. -L~/.rubies/ruby-2.5.1/lib -L. -L~/.rubies/ruby-2.5.1/lib  -fstack-protector -L/usr/local/lib     -lruby.2.5.1-static -framework Foundation -lxml2 -lpthread -lgmp -ldl -lobjc  "
checked program was:
/* begin */
1: #include "ruby.h"
2:
3: int main(int argc, char **argv)
4: {
5:   return 0;
6: }
/* end */

| pkg-config --cflags-only-I libxml-2.0
=> "-I/usr/include/libxml2\n"
| pkg-config --cflags-only-other libxml-2.0
=> "\n"
| pkg-config --libs-only-l libxml-2.0
=> "-lxml2\n"
package configuration for libxml-2.0
cflags:
ldflags:
libs: -lxml2

"pkg-config --exists libexslt"
| pkg-config --libs libexslt
=> "-lexslt -lxslt -lxml2 -lz -lpthread -licucore -lm -lxml2\n"
"clang -o conftest -I~/.rubies/ruby-2.5.1/include/ruby-2.5.0/x86_64-darwin18 -I~/.rubies/ruby-2.5.1/include/ruby-2.5.0/ruby/backward -I~/.rubies/ruby-2.5.1/include/ruby-2.5.0 -I. -I/usr/include/libxml2 -I/usr/local/opt/libxml2/include -I~/.rubies/ruby-2.5.1/include  -D_XOPEN_SOURCE -D_DARWIN_C_SOURCE -D_DARWIN_UNLIMITED_SELECT -D_REENTRANT    -O3 -Wno-error=shorten-64-to-32  -pipe   conftest.c  -L. -L~/.rubies/ruby-2.5.1/lib -L/usr/local/opt/libxml2/lib -L. -L~/.rubies/ruby-2.5.1/lib  -fstack-protector -L/usr/local/lib      -lxml2 -lruby.2.5.1-static -framework Foundation -lexslt -lxslt -lxml2 -lz -lpthread -licucore -lm -lxml2 -lpthread -lgmp -ldl -lobjc  "
checked program was:
/* begin */
1: #include "ruby.h"
2:
3: int main(int argc, char **argv)
4: {
5:   return 0;
6: }
/* end */

| pkg-config --cflags-only-I libexslt
=> "-I/usr/include/libxml2\n"
| pkg-config --cflags-only-other libexslt
=> "\n"
| pkg-config --libs-only-l libexslt
=> "-lexslt -lxslt -lxml2 -lz -lpthread -licucore -lm -lxml2\n"
package configuration for libexslt
cflags:
ldflags:
libs: -lexslt -lxslt -lxml2 -lz -lpthread -licucore -lm -lxml2

"clang -E -I~/.rubies/ruby-2.5.1/include/ruby-2.5.0/x86_64-darwin18 -I~/.rubies/ruby-2.5.1/include/ruby-2.5.0/ruby/backward -I~/.rubies/ruby-2.5.1/include/ruby-2.5.0 -I. -I/usr/include/libxml2 -I/usr/include/libxml2 -I/usr/local/opt/libxml2/include -I~/.rubies/ruby-2.5.1/include  -D_XOPEN_SOURCE -D_DARWIN_C_SOURCE -D_DARWIN_UNLIMITED_SELECT -D_REENTRANT    -O3 -Wno-error=shorten-64-to-32  -pipe     conftest.c -o conftest.i"
conftest.c:3:10: fatal error: 'libxml/xmlversion.h' file not found
#include <libxml/xmlversion.h>
         ^~~~~~~~~~~~~~~~~~~~~
1 error generated.
checked program was:
/* begin */
1: #include "ruby.h"
2:
3: #include <libxml/xmlversion.h>
/* end */

What operating system are you using?

macOS 10.14 Mojave

I've tried to reinstall everything what I could but without any success. I used pkg-config, libxml2 and libxslt from Homebrew...

[deepj]

I was successful with gem install nokogiri -- --use-system-libraries=true --with-xml2-include="$(xcrun --show-sdk-path)"/usr/include/libxml2

But not with libxml2 from Homebrew.

[flavorjones]

I'm sorry y'all are having trouble.

If someone who is a Mac user would care to verify that this works and submit a PR to the installation tutorial, I would be very grateful. Alternatively, if there's something we can change in the extconf, I'd appreciate a PR.

Also tagging @zenspider as he's been kind enough to offer assistance on behalf of Mac users in the past.

[zenspider]

I don't think the original report makes sense to me (reformatted):

Building native extensions with: '--use-system-libraries \
      --with-xslt-dir=/usr/local/opt/libxslt \
      --with-xslt-dir=/usr/local/opt/libxml2'

This says "build and link against system libraries, but look for their configs in these completely different builds". I would expect that to break. See this for more details:

http://www.nokogiri.org/tutorials/installing_nokogiri.html#using-your-system-libraries

[deepj]

@zenspider I put one of many iterations made by me here. The problem is still same if I just gem install nokogiri. It's not possible to install Nokogiri on Mojave without additional parameters.

You're right. There is a mistake in the command --with-xslt-dir=/usr/local/opt/libxml2' => --with-xml2-dir=/usr/local/opt/libxml2'
But the result is the same even I use the right form. Installation of Nokogiri failed on Mojave.

Only this worked

gem install nokogiri -- --use-system-libraries=true --with-xml2-include="$(xcrun --show-sdk-path)"/usr/include/libxml2

The expectation is I need only run gem install nokogiri on Mojave. There was no problems on the previous High Sierra.

[deepj]

OK. I found the reason why it's happening on Mojave

See https://developer.apple.com/documentation/xcode_release_notes/xcode_10_release_notes#3035624

It's needed to make another step and install this as a temporary workaround on Mojave:

open /Library/Developer/CommandLineTools/Packages/macOS_SDK_headers_for_macOS_10.14.pkg

For explanation see the Xcode release notes

[flavorjones]

@deepj Thank you for following up on the thread. Would you be so kind as to submit a PR to this file that includes steps other Mojave users can follow?

https://github.com/sparklemotion/nokogiri.org-tutorials/blob/master/content/installing_nokogiri.md

I unfortunately don't have a Mac and so can't verify the steps that I might write up are correct. I'd really appreciate your help!

[deepj]

@flavorjones I'll do. But what I got from the Xcode release notes, it's needed to change a way for looking headers on macOS 10.14 and later. Can @zenspider help here, please? I don't have no insight how these compilation stuffs work.

[dhh]

I just hit this same problem on a machine upgraded to 10.14.

[wsizoo]

Updating command line tools for Xcode fixed this for me.

Command Line Tools for Xcode 10.14

[flavorjones]

Again, an open request for Mac users to help support the Ruby community: Can someone please submit a PR to the installation tutorial explaining how to update xcode tooling on 10.14?

[cjxonix]

I was successful with gem install nokogiri -- --use-system-libraries=true --with-xml2-include="$(xcrun --show-sdk-path)"/usr/include/libxml2

But not with libxml2 from Homebrew.

This actually worked for on Mojav Mac OS X