Julia 1.9.3 and GMT 6.5.0, Wrapper GMT.jl Error

[visbilika]

Hi,

I am new to GMT with Julia, I have installed Julia 1.9.3 and GMT 6.5.0 in an openSUSE Linux environment.
But there is an error from Wrapper GMT.jl. The following is the error:

Fatal error: expression 'pthread_create(&threadId, &threadAttr, (void ()(void *))func, funcArg)' value 11 (expected zero)

[39735] signal (6.-6): Aborted
in expression starting at /home/virga/.julia/packages/GMT/1kBBX/src/GMT.jl:272
gsignal at /lib64/libc.so.6 (unknown line)
abort at /lib64/libc.so.6 (unknown line)

Then I switched directly from compiled Julia, with an error like this.

julia> using GMT
[ Info: Precompiling GMT [5752ebe1-31b9-557e-87aa-f909b540aa54]
ERROR: LoadError: ArgumentError: cannot convert NULL to string
Stacktrace:
  [1] unsafe_string
    @ ./strings/string.jl:84 [inlined]
  [2] get_PS
    @ ~/.julia/packages/GMT/GDTso/src/gmt_main.jl:586 [inlined]
  [3] GMTJL_Get_Object(API::Ptr{Nothing}, X::GMT.GMT_RESOURCE)
    @ GMT ~/.julia/packages/GMT/GDTso/src/gmt_main.jl:744
  [4] gmt(::String, ::GMT.GMTgrid{Float32, 2}, ::Vararg{Any})
    @ GMT ~/.julia/packages/GMT/GDTso/src/gmt_main.jl:176
  [5] finish_PS_module(::Dict{Symbol, Any}, ::Vector{String}, ::String, ::Bool, ::Bool, ::Bool, ::GMT.GMTgrid{Float32, 2}, ::Vararg{Any})
    @ GMT ~/.julia/packages/GMT/GDTso/src/common_options.jl:4189
  [6] grdcontour(cmd0::String, arg1::GMT.GMTgrid{Float32, 2}; first::Bool, kwargs::Base.Pairs{Symbol, Union{}, Tuple{}, NamedTuple{(), Tuple{}}})
    @ GMT ~/.julia/packages/GMT/GDTso/src/grdcontour.jl:138
  [7] grdcontour
    @ ~/.julia/packages/GMT/GDTso/src/grdcontour.jl:68 [inlined]
  [8] #grdcontour#334
    @ ~/.julia/packages/GMT/GDTso/src/grdcontour.jl:165 [inlined]
  [9] grdcontour(arg1::GMT.GMTgrid{Float32, 2})
    @ GMT ~/.julia/packages/GMT/GDTso/src/grdcontour.jl:165

Any help insight and solution would be much appreciated. Thank you.

[joa-quim]

Hi visbilika,
Sorry to hear this that seems a complicated case (and Suse Linux again). The first error seems to point into GMT building problem as there references to pthreads that comes from GLIB. Oddly it also says that the triggering line is one that include PrettyTables that should have nothing to do with pthreads.

You also said that you are new to GMT with Julia. Does it mean you are a GMT user and have GMT 6.4 installed in your computer? If yes we could try something first.

[visbilika]

Hi joa-quim thank you for your response,

The answer for your question is: Yes, I used GMT from my computer. When I use the GMT that is already installed on my computer, it results in a pthreads error.

And, I followed your first suggestion, I removed the library from the system, and the error changed as follows:

julia> using GMT
[ Info: Precompiling GMT [5752ebe1-31b9-557e-87aa-f909b540aa54]

libgomp: Thread creation failed: Resource temporarily unavailable
ERROR: Failed to precompile GMT [5752ebe1-31b9-557e-87aa-f909b540aa54] to "/home/virga/.julia/compiled/v1.9/GMT/jl_wGjX27".
Stacktrace:
 [1] error(s::String)
   @ Base ./error.jl:35
 [2] compilecache(pkg::Base.PkgId, path::String, internal_stderr::IO, internal_stdout::IO, keep_loaded_modules::Bool)
   @ Base ./loading.jl:2300
 [3] compilecache
   @ ./loading.jl:2167 [inlined]
 [4] _require(pkg::Base.PkgId, env::String)
   @ Base ./loading.jl:1805
 [5] _require_prelocked(uuidkey::Base.PkgId, env::String)
   @ Base ./loading.jl:1660
 [6] macro expansion
   @ ./loading.jl:1648 [inlined]
 [7] macro expansion
   @ ./lock.jl:267 [inlined]
 [8] require(into::Module, mod::Symbol)
   @ Base ./loading.jl:1611

Any help or insight would be greatly appreciated. Thank you.

[joa-quim]

Ah, I had not understood that first error came from a CLI run. So this is clearly a building problem. Did you build GMT or is it package installed version?

Anyway, can you please post this in the GMT forum where we can hopefully get more help? This looks like a Suse libs problem.

I followed your first suggestion, I removed the library from the system,

Sorry, some confusion. I didn't suggest to do that.

[visbilika]

For the CLI run issue is already solved.

I have tried to empty the LD_LIBRARY_PATH and the error changed as follows:

julia> using GMT
[ Info: Precompiling GMT [5752ebe1-31b9-557e-87aa-f909b540aa54]

libgomp: Thread creation failed: Resource temporarily unavailable
ERROR: Failed to precompile GMT [5752ebe1-31b9-557e-87aa-f909b540aa54] to "/home/virga/.julia/compiled/v1.9/GMT/jl_l1DuDF".
Stacktrace:
 [1] error(s::String)
   @ Base ./error.jl:35
 [2] compilecache(pkg::Base.PkgId, path::String, internal_stderr::IO, internal_stdout::IO, keep_loaded_modules::Bool)
   @ Base ./loading.jl:2300
 [3] compilecache
   @ ./loading.jl:2167 [inlined]
 [4] _require(pkg::Base.PkgId, env::String)
   @ Base ./loading.jl:1805
 [5] _require_prelocked(uuidkey::Base.PkgId, env::String)
   @ Base ./loading.jl:1660
 [6] macro expansion
   @ ./loading.jl:1648 [inlined]
 [7] macro expansion
   @ ./lock.jl:267 [inlined]
 [8] require(into::Module, mod::Symbol)
   @ Base ./loading.jl:1611

So, I export OMP_NUM_THREADS=1 and give results like this:

virga@cyclone:~> julia
               _
   _       _ ()_     |  Documentation: https://docs.julialang.org
  (_)     | (_) (_)    |
   _ _   | |  __ _   |  Type "?" for help, "]?" for Pkg help.
  | | | | | | |/ _` |  |
  | | |_| | | | (_| |  |  Version 1.9.3 (2023-08-24)
 / |\_'_|_|_|\__'_|  |
|__/                   |

But when I try to call using GMT the result is a segmentation fault error, as shown below:

julia> using GMT

julia> violin(rand(50),show=1)
Error: /rangecheck in --setlinecap--
Operand stack:
   3
Execution stack:
   %interp_exit   .runexec2   --nostringval--   setlinecap   --nostringval--   2   %stopped_push   --nostringval--   setlinecap   setlinecap   false   1   %stopped_push   1974   1   3   %oparray_pop   1973   1   3   %oparray_pop   1961   1   3   %oparray_pop   1817   1   3   %oparray_pop   --nostringval--   %errorexec_pop   .runexec2   --nostringval--   setlinecap   --nostringval--   2   %stopped_push   --nostringval--   1820   1   4   %oparray_pop   setlinecap
Dictionary stack:
   --dict:767/1123(ro)(G)--   --dict:0/20(G)--   --dict:75/200(L)--   --dict:155/250(L)--
Current allocation mode is local
Last OS error: No such file or directory
Current file position is 21086
psconvert [ERROR]: System call ['/home/virga/.julia/artifacts/4883a6685d86b068da934537a444a6d170d71730/bin/gs' -q -dNOSAFER -dNOPAUSE -dBATCH -sDEVICE=bbox -DPSL_no_pagefill -dMaxBitmap=2147483647 -dUseFastColor=true '/tmp/GMTjl_virga.ps' 2> './psconvert_10653c.bb'] returned error 256.

[10653] signal (11.1): Segmentation fault
in expression starting at REPL[2]:1
cfree at /lib64/libc.so.6 (unknown line)
PSL_endsession at /usr/local/atmos/lib64/libpostscriptlight.so.6 (unknown line)
gmt_end at /home/virga/.julia/artifacts/ec99c4bdc289bc1aa41d0609b33aa97d78687983/lib/libgmt.so (unknown line)
GMT_Destroy_Session at /home/virga/.julia/artifacts/ec99c4bdc289bc1aa41d0609b33aa97d78687983/lib/libgmt.so (unknown line)
GMT_Destroy_Session at /home/virga/.julia/packages/GMT/GDTso/src/libgmt.jl:17 [inlined]
gmt_restart at /home/virga/.julia/packages/GMT/GDTso/src/gmt_main.jl:234
gmt_restart at /home/virga/.julia/packages/GMT/GDTso/src/gmt_main.jl:234 [inlined]
resetGMT at /home/virga/.julia/packages/GMT/GDTso/src/gmt_main.jl:1296
gmt at /home/virga/.julia/packages/GMT/GDTso/src/gmt_main.jl:160
showfig at /home/virga/.julia/packages/GMT/GDTso/src/common_options.jl:3925
finish_PS_module at /home/virga/.julia/packages/GMT/GDTso/src/common_options.jl:4210
common_plot_xyz at /home/virga/.julia/packages/GMT/GDTso/src/psxy.jl:285
jl_apply at /home/virga/src/julia-1.9.3/src/julia.h:1880 [inlined]
do_apply at /home/virga/src/julia-1.9.3/src/builtins.c:730
helper2_violin at /home/virga/.julia/packages/GMT/GDTso/src/statplots.jl:674
#violin#754 at /home/virga/.julia/packages/GMT/GDTso/src/statplots.jl:507
violin at /home/virga/.julia/packages/GMT/GDTso/src/statplots.jl:499 [inlined]
#violin#753 at /home/virga/.julia/packages/GMT/GDTso/src/statplots.jl:491
violin at /home/virga/.julia/packages/GMT/GDTso/src/statplots.jl:488 [inlined]
violin at /home/virga/.julia/packages/GMT/GDTso/src/statplots.jl:488
unknown function (ip: 0x150e07ef0e66)
jl_apply at /home/virga/src/julia-1.9.3/src/julia.h:1880 [inlined]
do_call at /home/virga/src/julia-1.9.3/src/interpreter.c:126
eval_value at /home/virga/src/julia-1.9.3/src/interpreter.c:226
eval_stmt_value at /home/virga/src/julia-1.9.3/src/interpreter.c:177 [inlined]
eval_body at /home/virga/src/julia-1.9.3/src/interpreter.c:624
jl_interpret_toplevel_thunk at /home/virga/src/julia-1.9.3/src/interpreter.c:762
jl_toplevel_eval_flex at /home/virga/src/julia-1.9.3/src/toplevel.c:912
jl_toplevel_eval_flex at /home/virga/src/julia-1.9.3/src/toplevel.c:856
ijl_toplevel_eval_in at /home/virga/src/julia-1.9.3/src/toplevel.c:971
eval at ./boot.jl:370 [inlined]
eval_user_input at /home/virga/src/julia-1.9.3/usr/share/julia/stdlib/v1.9/REPL/src/REPL.jl:153
repl_backend_loop at /home/virga/src/julia-1.9.3/usr/share/julia/stdlib/v1.9/REPL/src/REPL.jl:249
#start_repl_backend#46 at /home/virga/src/julia-1.9.3/usr/share/julia/stdlib/v1.9/REPL/src/REPL.jl:234
start_repl_backend at /home/virga/src/julia-1.9.3/usr/share/julia/stdlib/v1.9/REPL/src/REPL.jl:231
#run_repl#59 at /home/virga/src/julia-1.9.3/usr/share/julia/stdlib/v1.9/REPL/src/REPL.jl:379
run_repl at /home/virga/src/julia-1.9.3/usr/share/julia/stdlib/v1.9/REPL/src/REPL.jl:365
jfptr_run_repl_61288 at /home/virga/julia-1.9.3/lib/julia/sys.so (unknown line)
#1017 at ./client.jl:421
jfptr_YY.1017_34677 at /home/virga/julia-1.9.3/lib/julia/sys.so (unknown line)
jl_apply at /home/virga/src/julia-1.9.3/src/julia.h:1880 [inlined]
jl_f__call_latest at /home/virga/src/julia-1.9.3/src/builtins.c:774
#invokelatest#2 at ./essentials.jl:819 [inlined]
invokelatest at ./essentials.jl:816 [inlined]
run_main_repl at ./client.jl:405
exec_options at ./client.jl:322
_start at ./client.jl:522
jfptr__start_46705 at /home/virga/julia-1.9.3/lib/julia/sys.so (unknown line)
jl_apply at /home/virga/src/julia-1.9.3/src/julia.h:1880 [inlined]
true_main at /home/virga/src/julia-1.9.3/src/jlapi.c:573
jl_repl_entrypoint at /home/virga/src/julia-1.9.3/src/jlapi.c:717
main at julia (unknown line)
__libc_start_main at /lib64/libc.so.6 (unknown line)
_start at julia (unknown line)
Allocations: 5568953 (Pool: 5566355; Big: 2598); GC: 7
Segmentation fault (core dumped)

I think the GMT binary has been successfully compiled, but there is a library issue causing conflicts during the process.

Anyway, can you please post this in the GMT forum where we can hopefully get more help? This looks like a Suse libs problem.

Your input is greatly appreciated. Thank you.

[visbilika]

Ah, I had not understood that first error came from a CLI run. So this is clearly a building problem. Did you build GMT or is it package installed version?

I'm using GMT package installed version.

[joa-quim]

For the CLI run issue is already solved.

How?

I have tried to empty the LD_LIBRARY_PATH and the error changed as follows:

Keeps being strange. That I know, the LD_LIBRARY_PATH contents should not matter in the Julia side.

The GMT.jl error starts in ghostscript, which is very strange again. What happens if you do this:

julia> violin(rand(50), name="viol.ps")

and than, with the CLI move to where the viol.ps was saved and do

gmt psconvert viol.ps -A -Tg

Do you get a viol.png file?

[visbilika]

The GMT.jl error starts in ghostscript, which is very strange again. What happens if you do this:

 julia> violin(rand(50), name="viol.ps") 

I have attempted this and success, as well as this part:

and than, with the CLI move to where the viol.ps was saved and do

gmt psconvert viol.ps -A -Tg

The end result is that I have obtained a .png file and successfully installed GMT.jl in Julia. Thank you.

But, when attempting to install this plugin for wind barb plots from the following source: https://sites.google.com/site/masahigaki/gmt/gmt-windbarb?authuser=0, I encountered a new error. I would greatly appreciate your guidance on how to successfully add or install the plugin in GMT.jl. Thank you.

[joa-quim]

The end result is that I have obtained a .png file and successfully installed GMT.jl in Julia. Thank you.

But this was just a test to see if the problem was in the ghostscript in Julia. And apparently is, which I can't understand.
If the this problems persists then your GMT.jl installation is more or less useless.

Regarding the plugin, which looks very cool and we had never heard about it in the GMT dev, the case is not straightforward but there is some hope. I need to learn more before I can answer you.

[joa-quim]

Do you have "masahigaki" contact? We are considering adding the windbarbs package as a GMT supplement and would like to invite him to be the maintainer.

Regarding its usage from GMT.jl, it needs first to be built within GMT. I have a working prototype but so far for Windows only.

Are you a meteorologist that want's to make windbarb maps?

[visbilika]

Dear joa-quim, thank you for your answer.

Do you have "masahigaki" contact? We are considering adding the windbarbs package as a GMT supplement and would like to invite him to be the maintainer.

I found 'masahigaki' contact at e-mail: higamasa@dream.com and m-higaki@naps.kishou.go.jp. But I have not contacted him yet for this issues.

Are you a meteorologist that want's to make windbarb maps?

Yes, I am meteorologist from Indonesia, I want to plot or make windbarb map natively from GMT.

For your information, I have successfully installed the windbarb plugins for GMT in Julia on both Mac and Linux. However, my next challenge is how to call these plugins from Julia.

Thank you so much.

[joa-quim]

Thanks for finding masahigaki contact.

To use GMT in Julia one must understand some facts. GMT.jl in unix uses a pre-build GMT binary so your windbarb GMT build is ignored by GMT.jl.

However, for situations like this I created a workaround but I cannot guarantee that it still works, but please try it. FRom the notes I left in the code the recipe is the following:

# The file deps/deps.jl is created by compile from the deps/build.jl
# On Windows we use a system wide GMT if it is found from path or install it from a GMT installer. It is a MSVC binary.
# On Unix the default is to use the GMT_jll artifact. However this can be changed to use a system wide GMT installation.
# To swap to a system wide GMT installation, do (in REPL):
# 1- ENV["SYSTEMWIDE_GMT"] = 1;
# 2- import Pkg; Pkg.build("GMT")
# 3- restart Julia
#
# Note the above will work up until some other reason triggers a Julia recompile, where the JLL artifacts 
# will be used again. To make the ENV["SYSTEMWIDE_GMT"] = 1 solution permanent, declare a "SYSTEMWIDE_GMT"
# environment variable permanently in your .bashrc (or whatever).

Next is the question of using it. I have committed a change that introduced a GMT.jl windbarbs function, but it needs the GMT.jl master version. I don't know if you are familiar with this git talk, but you can update your GMT.jl to master version with

] ad GMT#master

next, typing ``? windbarbs` should show you the, nearly nonexistent, usage help. If you get this far, you are at a blink to use it.

All will be much simpler when the GMT-built-by-the-Julia-mechinery (the GMT_jll artifact) gets done.

[visbilika]

Thank you for your suggestions and advice.

I have run your advice step by step, modified GMT.jl according to the changes made in its source git, and run a new GMT test. The result is an error, as shown below:

julia> G = gmt("grdmath -R-2/2/-2/2 -I0.1 X Y R2 NEG EXP X MUL")
title: Produced by grdmath
command: gmt grdmath -R-2/2/-2/2 -I0.1 X Y R2 NEG EXP X MUL =@GMTAPI@-S-O-G-G-G-N-000018
Gridline node registration used
x_min: -2.0 x_max :2.0  x_inc :0.1  n_columns :41
y_min: -2.0 y_max :2.0  y_inc :0.1  n_rows :41
z_min: -0.42883849143981934    z_max :0.42883849143981934
Mem layout: BCB

julia> dzdy = gmt("grdmath ? DDY", G);

julia> dzdx = gmt("grdmath ? DDX", G);

julia> windbarbs(dzdx, dzdy, Vd=2)
Warning: the following options were not consumed in grdvector => [:barbs]
ERROR: MethodError: no method matching similar(::Char, ::Type{Any})

Closest candidates are:
  similar(::Union{LinearAlgebra.Adjoint{T, var"#s972"}, LinearAlgebra.Transpose{T, var"#s972"}} where {T, var"#s972"<:(AbstractVector)}, ::Type{T}) where T
   @ LinearAlgebra /Applications/Julia-1.9.3.app/Contents/Resources/julia/share/julia/stdlib/v1.9/LinearAlgebra/src/adjtrans.jl:330
  similar(::Union{LinearAlgebra.Adjoint{T, S}, LinearAlgebra.Transpose{T, S}} where {T, S}, ::Type{T}) where T
   @ LinearAlgebra /Applications/Julia-1.9.3.app/Contents/Resources/julia/share/julia/stdlib/v1.9/LinearAlgebra/src/adjtrans.jl:334
  similar(::Union{LinearAlgebra.Adjoint{T, S}, LinearAlgebra.Transpose{T, S}} where {T, S}, ::Type{T}, ::Tuple{Vararg{Int64, N}}) where {T, N}
   @ LinearAlgebra /Applications/Julia-1.9.3.app/Contents/Resources/julia/share/julia/stdlib/v1.9/LinearAlgebra/src/adjtrans.jl:335
  ...

Stacktrace:
 [1] _similar_or_copy(x::Char, #unused#::Type{Any})
   @ Base ./set.jl:572
 [2] replace(A::Char, old_new::Pair{String, String}; count::Nothing)
   @ Base ./set.jl:690
 [3] replace(A::Char, old_new::Pair{String, String})
   @ Base ./set.jl:686
 [4] windbarbs(arg1::GMTgrid{Float32, 2}, arg2::GMTgrid{Float32, 2}; first::Bool, kwargs::Base.Pairs{Symbol, Int64, Tuple{Symbol}, NamedTuple{(:Vd,), Tuple{Int64}}})
   @ GMT ~/.julia/packages/GMT/AjFsi/src/windbarbs/windbarbs.jl:46
 [5] top-level scope
   @ REPL[13]:1

And for the second test, give an error:

julia> windbarbs([0 90 0 45; 10 90 10 45], R=:g, Vd=2)
"psbarb  -Rg -JX15c/10c -Baf -BWSen -P -K > /var/folders/_4/ly31qk1934s924zy4n94zj7h0000gn/T/GMTjl_jpatiani.ps"

So, the installation result of the windbarbs.so plugins in GMT.jl has been recognized in Julia's GMT. However, during the test run, there was still an error.

I would like to ask about how to verify whether the plugins have been detected and can be loaded in GMT.jl ?

When I execute this step, it results in a GMT [ERROR], as shown below:

julia> windbarbs([0 90 0 45; 10 90 10 45], R=:g, show=1)
GMT [ERROR]: Shared GMT module not found: psbarb

===  GMT core: The main modules of the Generic Mapping Tools  ===

when I checked the GMT module,

julia> run(`gmt --show-modules`) 

I found the following result:

…
polar
polespotter
psbarb
rotconverter
rotsmoother
...

There appears psbarb module

[joa-quim]

OK, several things. You probably need to update GMT.jl. (] up). At the end the gmtvector.jl must have this line.

The Vd=2 is a debug option that only prints the string that will be send to GMT but does not execute it. Remember that the CI virtual machines do not have a GMT built with the windbarbs new C code. For example, I get

julia> windbarbs(dzdx, dzdy, Vd=2)
"grdbarb  -JX15c/0 -Baf -BWSen -Ve -Ix4/4  -W0.5 -P -K > c:\\TMP/GMTjl_j.ps

Now, this is bad.

julia> windbarbs([0 90 0 45; 10 90 10 45], R=:g, show=1)
GMT [ERROR]: Shared GMT module not found: psbarb

It means the barbs supplement of your GMT build is not working, and think I know why. The original code was not exporting the GMT_psbarb and GMT_grdbarb functions so they were not added to the windbarbs.so shared lib. At least that's how I see it from Windows. The only solution to this that I see is that you rebuild GMT, but this time you use the windbarbs-suppl branch of the GMT dev source tree. That is, you need to check out GMT from Github, change to that branch and build GMT normally. Since you already built GMT once, this shouldn't be very difficult. That is how I build the version that works for me.

Ant, to finish by now. The option that runs from a table (the equivalent of psbarb) seems to be having some issues because it now plots only a blank map, but it used to work. Need to investigate that too.

Good luck

[visbilika]

Thank you for your advice and suggestion,

I have successfully installed GMT.jl with the windbarbs function, and I have installed GMT from the windbarb-suppl branch of the GMT development source tree, allowing me to make windbarb maps on Macs. For the next step, I plan to install and follow your suggestions on Linux. I am hopeful that there will be no further issues in the next steps.

Thank you very much, Joa-quim.

[joa-quim]

Glad to year that it worked. Meanwhile I have released a GMT.jl 1.6.0 that contains the windbarbs function. So, no more need to use the master version.

In case that you produce a kind of official weather maps with windbarbs made with GMT, please consider posting one of those maps in the GMT forum under the Showcase category.