volume.nii

[timholy]

At one point you mentioned a replacement for ImageView based on GLVisualize; this is of considerable interest to me. I'm guessing that the volume.jl and related examples are what you mean?

One immediate issue: there doesn't appear to be a downloader for volume.nii. I tried searching for it, but the file I found isn't of the right size and consequently I get

ERROR: LoadError: BoundsError: attempt to access 1024×1024×4 Array{Float64,3} at index [Colon(),Colon(),100]
 in throw_boundserror(::Array{Float64,3}, ::Tuple{Colon,Colon,Int64}) at ./abstractarray.jl:363
 in checkbounds at ./abstractarray.jl:292 [inlined]
 in _getindex at ./multidimensional.jl:270 [inlined]
 in getindex(::Array{Float64,3}, ::Colon, ::Colon, ::Int64) at ./abstractarray.jl:760
 in include_from_node1(::String) at ./loading.jl:488
while loading /home/tim/.julia/v0.5/GLPlot/example/volume.jl, in expression starting on line 17

If you can tell me where that file comes from, I can contribute an installer and tweak the paths in the various examples.

[timholy]

BTW, did you see this? http://julialang.org/soc/projects/images.html

I took the liberty of listing you, hope that's OK.

[SimonDanisch]

Excellent!! :) I wanted to do the same, but didn't get to it!

[SimonDanisch]

Are you talking about?
https://github.com/SimonDanisch/GLPlot.jl/blob/68c9b217d2cc9b5814ecfd91f60f36f28bb6031f/example/volume.jl ?
That's just some random file from a friend who scanned her brain ;) If you have any demo file we can use and automatically download it, that would obviously much better!

[timholy]

I got one from http://www.humanconnectome.org/data/ (a project run out of my home institution). It's 68M, so if we use it there seems to be a hosting issue we'd have to solve. Conceivably we could store it as Float16 (it's now Float32) and at lower resolution (it's now (260,311,260)), but there are of course costs to pay especially in reducing the resolution.

But without dealing with that, I don't even understand what the buttons on the left mean. Aside from rotating the head, how does one interact with the image? Suppose I just want to "play through" each 2d slice? Finally, whatever the code after the gui command is doing, it actually messes up the slice images.

[timholy]

(You need a free account to access data on HCP, so it's not great for automatic downloads.)

[timholy]

Pinging @vanessen in case he has any suggestions. David, this repository is a high-performance OpenGL based visualization suite for Julia. Main page is at https://github.com/JuliaGL/GLVisualize.jl.

[SimonDanisch]

Yeah, that example is a bit buggy, admittedly!
I should probably rewrite it to make it work better!
But this is how it should look:
https://vimeo.com/180307247

[timholy]

That explains it, here's the window I see right after running the example script (switching to the dataset I downloaded from HPC):

But prompted by this, I noticed the little scrollbar to the right; once I grab it and move it, the window suddenly reshapes and I see this:

The sliders work very well, although I note a few problems (of which some might come from using a different dataset than you):

• the "zoom" on the 2d slice images does not encompass the entire dataset (comparing against ImageView). In fact, it's dependent on the window size, and seems to get worse if I make the window bigger
• I get a strange rectangle in the lower-left panel
• when I try to expand the upper controls, I see the following:

which seems quite different than what you show in that movie.

• trying to expand the topmost control gives me an error in the REPL:

julia> Failed to push!
    false
to node
    Signal{Bool}(false, nactions=2)
MethodError: no method matching _default(::Float32, ::GLAbstraction.Style{:default}, ::Dict{Symbol,Any})
Closest candidates are:
  _default(::GeometryTypes.HomogenousMesh{FixedSizeArrays.Point{3,Float32},GeometryTypes.Face{3,UInt32,-1},GeometryTypes.Normal{3,Float32},Void,ColorTypes.RGBA{Float32},Void,Void}, ::GLAbstraction.Style{StyleValue}, ::Dict{K,V}) at /home/tim/.julia/v0.5/GLVisualize/src/visualize/mesh.jl:40
  _default{T<:FixedSizeArrays.Point{N,T}}(::Union{Array{GeometryTypes.Simplex{2,T<:FixedSizeArrays.Point},1},GLAbstraction.GPUArray{GeometryTypes.Simplex{2,T<:FixedSizeArrays.Point},1},Reactive.Signal{Array{GeometryTypes.Simplex{2,T<:FixedSizeArrays.Point},1}}}, ::GLAbstraction.Style{StyleValue}, ::Dict{K,V}) at /home/tim/.julia/v0.5/GLVisualize/src/visualize/lines.jl:107
  _default{T<:ColorTypes.Colorant{T,N},X}(::Union{ImageMetadata.ImageMeta{T<:ColorTypes.Colorant,2,X},Reactive.Signal{ImageMetadata.ImageMeta{T<:ColorTypes.Colorant,2,X}}}, ::GLAbstraction.Style{StyleValue}, ::Dict{K,V}) at /home/tim/.julia/v0.5/GLVisualize/src/visualize/image_like.jl:4
  ...
 in default(::Any, ::Any, ::Any) at /home/tim/.julia/v0.5/GLVisualize/src/visualize_interface.jl:4
 in visualize(::Any, ::GLAbstraction.Style{:default}, ::Dict{Symbol,Any}) at /home/tim/.julia/v0.5/GLVisualize/src/visualize_interface.jl:21
 in (::GLVisualize.#kw##visualize)(::Array{Any,1}, ::GLVisualize.#visualize, ::Any, ::Symbol) at ./<missing>:0
 in #choice_visual#162(::Array{Any,1}, ::Function, ::Float32) at /home/tim/.julia/v0.5/GLVisualize/src/gui/options.jl:25
 in (::GLVisualize.#kw##choice_visual)(::Array{Any,1}, ::GLVisualize.#choice_visual, ::Float32) at ./<missing>:0
 in (::GLVisualize.##167#170{Int64,Int64,Tuple{Int64,Int64},Array{Any,1}})(::Tuple{Int64,Float32}) at /home/tim/.julia/v0.5/GLVisualize/src/gui/options.jl:59
 in collect(::Base.Generator{Enumerate{Array{Float32,1}},GLVisualize.##167#170{Int64,Int64,Tuple{Int64,Int64},Array{Any,1}}}) at ./array.jl:307
 in #widget#166(::Int64, ::Int64, ::Tuple{Int64,Int64}, ::Array{Any,1}, ::Function, ::Reactive.Signal{Array{Float32,1}}, ::GLWindow.Screen) at /home/tim/.julia/v0.5/GLVisualize/src/gui/options.jl:57
 in (::GLVisualize.#kw##widget)(::Array{Any,1}, ::GLVisualize.#widget, ::Reactive.Signal{Array{Float32,1}}, ::GLWindow.Screen) at ./<missing>:0
 in #extract_edit_menu#175(::Int64, ::Int64, ::Function, ::Array{Pair{Symbol,Any},1}, ::GLWindow.Screen, ::Reactive.Signal{Bool}) at /home/tim/.julia/v0.5/GLVisualize/src/gui/container.jl:33
 in (::GLPlot.##54#58{GLAbstraction.RenderObject{GLVisualize.##33#35{GLVisualize.VolumePrerender}}})(::Tuple{Bool,Int64}, ::Bool) at /home/tim/.julia/v0.5/GLPlot/src/plot.jl:164
 in (::Reactive.##25#26{GLPlot.##54#58{GLAbstraction.RenderObject{GLVisualize.##33#35{GLVisualize.VolumePrerender}}},Tuple{Reactive.Signal{Bool}}})(::Reactive.Signal{Tuple{Bool,Int64}}, ::Int64) at /home/tim/.julia/v0.5/Reactive/src/operators.jl:114
 in do_action at /home/tim/.julia/v0.5/Reactive/src/core.jl:141 [inlined]
 in run(::Int64) at /home/tim/.julia/v0.5/Reactive/src/core.jl:218
 in poll_reactive at /home/tim/.julia/v0.5/GLWindow/src/render.jl:31 [inlined]
 in glplot_renderloop(::GLWindow.Screen, ::Reactive.Signal{Bool}, ::Reactive.Signal{Bool}) at /home/tim/.julia/v0.5/GLPlot/src/screen.jl:65
 in (::GLPlot.##74#84)() at ./task.jl:360

Despite the list of problems, I am very intrigued; it's quite a functional GUI for ~20 lines of user code! Happy to work with you to try to fix the bugs, but I might need some assistance/guidance since I'm not entirely sure what's intended and what's not.

[SimonDanisch]

Oh yeah please bug me with these before you try to debug something I very well know about! ;)
Most of what you describe is the side effect of Plots.jl not working nicely with GLPlot. I think what I will do is to port most of the functionality of Plots to GLPlot, and make it all natively available!
Let me see if I can update the example to make it behave a bit better!

[SimonDanisch]

Here you go: 92ffcb6
The axes are gone because Plots.jl created them. I need to write up some API to nicely add one (problem: axis should be camera and maybe even content aware!)

trying to expand the topmost control gives me an error in the REPL:

I think this is an old error... Try checking out GLVisualize!

[timholy]

I think what I will do is to port most of the functionality of Plots to GLPlot, and make it all natively available!

I think that's a good idea. Plots is amazing, but since it has to be a wrapper around many toolkits it might (in the long run) hold you back.

And checking out master on GLVisualize and using your new commit: that's a lot better! All of those bugs I mentioned above are gone.

The main remaining problem is (at least for this dataset) the perspective changes in the top-left panel (when I click-drag with mouse) are relative to a corner of the image rather than the center, so the head pivots crazily around the image. I'm also struggling a bit with how to get what I want from the render settings; any documentation on the meaning of the parameters and how they affect the image?

[SimonDanisch]

I hope it helps to play around with it in the edit menu!

I also half heartedly implemented this function:

"""
you can call get_docs exactly like glplot, but get back a printed representation of all parameters.
If you're lucky, they will even be accompanied by some explanation! 
"""
get_docs(rand(Float32, 3,3,3)) # e.g. for getting help with a volume

[SimonDanisch]

as a reference: https://github.com/JuliaGL/GLVisualize.jl/blob/master/src/documentation.jl#L43

[SimonDanisch]

This should loosely map to this macro:
https://github.com/JuliaGL/GLVisualize.jl/blob/master/src/visualize/image_like.jl#L47
That macro makes up the default values and as you can see, also allow me to connect the parameter with a doc string.

[timholy]

The main remaining problem is (at least for this dataset) the perspective changes in the top-left panel (when I click-drag with mouse) are relative to a corner of the image rather than the center, so the head pivots crazily around the image.

I should have pointed out that a good way to specify the origin is with "non-1" indexing. In ImageFiltering I now have a centered function that provides default indices:

julia> x = (-9:9)/3;

julia> y = ImageFiltering.centered(exp(-x.^2/2))
OffsetArrays.OffsetArray{Float64,1,Array{Float64,1}} with indices -9:9:
 0.011109 
 0.0285655
 0.0657285
 0.135335 
 0.249352 
 0.411112 
 0.606531 
 0.800737 
 0.945959 
 1.0      
 0.945959 
 0.800737 
 0.606531 
 0.411112 
 0.249352 
 0.135335 
 0.0657285
 0.0285655
 0.011109 

One could allow the user to communicate the intended center that way.

I hope it helps to play around with it in the edit menu!

It does, but with the data set I have I'm noting the following:

• when the window first opens, I get this:
  
  but then as soon as I drag the scrollbar and get the right panel, then expand the topmost menu I get
  
  Note how the isosurface rendering changed. By tweaking isorange I can at least get close to what it was showing at the beginning.

• for images with a certain amount of noise, isosurfaces are problematic. absorption and maxproject should be less sensitive to pixel-by-pixel noise. When I switch to absorption, then the whole panel switches to white, with a faint gray color surrounding the head:
  
  As you can probably tell, I dragged some of the colormap settings as well as playing with isovalue and isorange; I have yet to observe any effect of tweaking those parameters.

• when I switch to maxprojection I get much more sensible behavior. Is there a way to click and restore the default color_map? It was all out of whack from my experiments.

This is still really cool. How hard would it be to set up perpendicular viewing axes and have the head rotation affect the slice orientation? Would it help to have an AbstractArray type like the SlicedPlane type I mentioned in my JuliaCon talk?