[SCIFIO] Bio-Formats Imaris writer

[Henry Pinkard]

Hi Mark,

I tried out the ETH-Zurich distribution and it worked! I don't have the
slightest idea *how* it worked though, so I am quite curious if you could
shed some light on this.

In any case, I made a pull request to add the writer to SCIFIO. There's
still some improvements to be made with time stamps, etc., but I think the
writer will already be quite useful to many people as is.

Thanks for all your help!

Henry

On Sun, Jan 25, 2015 at 5:30 AM, Mark Hiner <hiner at wisc.edu> wrote:

> Hi Henry,
>
>  One last note on this topic - Tobias Pietzsch pointed out that ETH-Zurich
> has an HDF5 Java distribution currently in use in the BigDataViewer[1].
> There's a wiki[2] and artifacts on the ImageJ repository[3] if you'd like
> to try it.
>
>
> [1] http://fiji.sc/BigDataViewer
> [2] https://wiki-bsse.ethz.ch/pages/viewpage.action?pageId=26609113
> [3] http://maven.imagej.net/index.html#nexus-search;quick~cisd
>
> On Sat, Jan 24, 2015 at 5:20 AM, Mark Hiner <hiner at wisc.edu> wrote:
>
>> Hi all,
>>
>>  I was able to finish the (now renamed) nar-hdf5[1] for Windows. A 0.1.0
>> release is now available from:
>>
>> <repository>
>>   <id>imagej.public</id>
>>   <url>http://maven.imagej.net/content/groups/public</url>
>> </repository>
>>
>> The dependency GAVs are:
>>
>> <dependency>
>>   <groupId>ncsa.hdf</groupId>
>>   <artifactId>nar-hdf5</artifactId>
>>   <version>0.1.0</version>
>> </dependency>
>>
>> <dependency>
>>   <groupId>ncsa.hdf</groupId>
>>   <artifactId>nar-hdf5-jni</artifactId>
>>   <version>0.1.0</version>
>>   <classifier>amd64-windows-msvc</classifier>
>>   <type>nar</type>
>> </dependency>
>>
>> <dependency>
>>   <groupId>ncsa.hdf</groupId>
>>   <artifactId>nar-hdf5-noarch</artifactId>
>>   <version>0.1.0</version>
>>   <type>nar</type>
>> </dependency>
>>
>> Hope that helps.
>>
>> Best,
>> Mark
>>
>> [1] https://github.com/scijava/nar-hdf5
>>
>> On Fri, Jan 23, 2015 at 10:25 AM, Mark Hiner <hiner at wisc.edu> wrote:
>>
>>> Hi Henry,
>>>
>>> >
>>> http://web.archive.org/web/20120627025240/http://www.buildanddeploy.com/node/14
>>> >
>>> http://web.archive.org/web/20120308042202/http://www.buildanddeploy.com/node/17
>>>
>>> Thanks for sharing these links, they were helpful. Since you already
>>> have .jars with the native code and jni code, I would be happy to upload
>>> them to our Maven repository so they can be consumed. This would at least
>>> allow others to build and test your format.
>>>
>>> The major limitation of this method is that requiring Maven
>>> configuration to extract the HDF5 code and get it on the java.library.path
>>> is not extensible. I believe it would prevent your format from being used
>>> dynamically, and it's not really a technique other 3rd party developers can
>>> use, because it requires updating pom configuration.
>>>
>>> The reason I was advocating for using the nar-maven-plugin is that,
>>> ideally, the resulting archive created would not need to be unpacked and
>>> would automatically add the native code to the java.library.path[1]. So
>>> using your format as a flagship nar-maven-plugin would a) make it more
>>> awesome and b) serve as a guide for future format developers.
>>>
>>> It is, however, certainly more work to use the nar-maven-plugin right
>>> now. So, if you can send me your .jars with native and JNI code, I'll try
>>> to get them on our Maven shortly.
>>>
>>> Best,
>>> Mark
>>>
>>> P.S. I did start a project to NAR HDF5[2] but it's not functional yet.
>>>
>>> [1]
>>> http://maven-nar.github.io/usage.html#Create_a_JNI_library_that_is_automatically_unpacked_from_the_.nar_file_using_the_native-lib-loader_project
>>> [2] https://github.com/scijava/hdf-jni/tree/create-nar
>>>
>>> On Fri, Jan 16, 2015 at 1:53 PM, Henry Pinkard <henry.pinkard at gmail.com>
>>> wrote:
>>>
>>>> Hi Mark,
>>>>
>>>> Okay, I'm following four tickets for LUTs and timestamps and I can
>>>> update the writer accordingly once those are settled. Thanks for that.
>>>>
>>>> In the meantime, I looked into the native libraries issue a bit more.
>>>> The purpose of the NAR plugin seems to be compiling native code and adding
>>>> them to archives, but since I already have the platform specific binaries,
>>>> I think there is a simpler way of doing this that doesn't rely on having to
>>>> compile the native code, which you can read more about here:
>>>>
>>>>
>>>> http://web.archive.org/web/20120627025240/http://www.buildanddeploy.com/node/14
>>>>
>>>> http://web.archive.org/web/20120308042202/http://www.buildanddeploy.com/node/17
>>>>
>>>> Basically, the idea is:
>>>> 1) Add the third party .jar with JNI code that calls native binaries as
>>>> a dependency
>>>> 2) Package native binaries (.dll, .so, or .jnilib) into separate .jar
>>>> files specific to each OS/architecture.
>>>> 3) Add these .jar's to the SCIFIO maven repository with a classifier
>>>> flag indicating OS/arch.
>>>> 4) Add dependencies that get the correct jar based on OS/arch.
>>>> 5) Use maven dependency plugin to unpack these libraries at build time.
>>>> 6) Use maven surefire plugin to set java.library.path so that native
>>>> libraries can be found at run time.
>>>>
>>>> I think much of this needs to happen from your end (but this is my
>>>> first time using maven, so correct me if I'm wrong). I've created the
>>>> .jar's holding the native code, and I have the .jar with JNI code as well
>>>> that I can pass along.
>>>>
>>>> What do you think?
>>>>
>>>> Best,
>>>> Henry
>>>>
>>>>
>>>> On Fri, Jan 16, 2015 at 9:20 AM, Mark Hiner <hiner at wisc.edu> wrote:
>>>>
>>>>> Hi Henry,
>>>>>
>>>>> >Can you offer up any examples of how to extract date/time metadata
>>>>> from image planes?
>>>>>
>>>>> Of course, sorry I forgot these in my last e-mail.
>>>>>
>>>>> Basically time stamps can be incorporated into Metadata by using the
>>>>> net.imagej.axis.LinearAxis API to set the origin and scale[1], and
>>>>> CalibratedAxis to set the units[2] on the time axis (Axes.TIME) of the
>>>>> ImageMetadata.
>>>>>
>>>>> However, this process is not automated yet so only Formats which
>>>>> explicitly set this information would have time stamp information, and
>>>>> there is no API in the Plane to convert its plane index to a time stamp.
>>>>> So, I opened another issue to get these populated automatically[3] - at
>>>>> which point a Plane object should just be able to tell you its time/date
>>>>> stamp, for any core SCIFIO format that records this information.
>>>>>
>>>>> >My TIFF dataset was simply one of the ImageJ samples. I made it using
>>>>> a pre-IJ2 version of FIJI by selecting File--Open samples--Mitosis and
>>>>> saving as a TIFF.
>>>>> >I then reopened it using SCIFIO.
>>>>>
>>>>> Great, thanks. I'll do this and look into why the LUTs aren't
>>>>> populated.
>>>>>
>>>>> Best,
>>>>> Mark
>>>>>
>>>>> [1]
>>>>> https://github.com/imagej/imagej-common/blob/1c96184fda188eba9d56d87364e141dbf07a286d/src/main/java/net/imagej/axis/LinearAxis.java#L44-50
>>>>> [2]
>>>>> https://github.com/imagej/imagej-common/blob/master/src/main/java/net/imagej/axis/CalibratedAxis.java#L47
>>>>> [3] https://github.com/scifio/scifio/issues/240
>>>>>
>>>>> On Thu, Jan 15, 2015 at 5:27 PM, Henry Pinkard <
>>>>> henry.pinkard at gmail.com> wrote:
>>>>>
>>>>>> Hi Mark,
>>>>>>
>>>>>> Can you offer up any examples of how to extract date/time metadata
>>>>>> from image planes?
>>>>>>
>>>>>>
>>>>>> On Thu, Jan 15, 2015 at 4:21 PM, Mark Hiner <hiner at wisc.edu> wrote:
>>>>>>
>>>>>>> Hi Henry,
>>>>>>>
>>>>>>> > Is it possible to include these binaries directly in the maven
>>>>>>> repository?
>>>>>>>
>>>>>>> Not directly. But since their license allows them to be
>>>>>>> redistributed, I think what we want is to make NARs out of them[1] and put
>>>>>>> the NAR in the maven repository.
>>>>>>>
>>>>>>> However, there are almost certainly limitations right now. I
>>>>>>> personally do not have experience with the nar-maven-plugin, but I would go
>>>>>>> into it expecting some hurdles.
>>>>>>>
>>>>>>> On the other hand, I think your ImarisFormat is an awesome use case
>>>>>>> for the nar-maven-plugin and this is a great opportunity to work out
>>>>>>> problems and document NAR usage in SCIFIO/ImageJ2/Fiji! So I'd be happy to
>>>>>>> help investigate further.
>>>>>>>
>>>>>>
>>>>>> Okay, I'll play around with the NAR library and see what I come up
>>>>>> with.
>>>>>>
>>>>>>
>>>>>>
>>>>>>> Sort of. I think the SCIFIOConfig and Writer.getColorModel API are
>>>>>>> incorrect to use java.awt.image.ColorModels. I opened a ticket[4] to update
>>>>>>> them to the ImgLib2 ColorTable class. When that ticket is complete, this is
>>>>>>> how everything should work:
>>>>>>>
>>>>>>> * You should make an ImarisFormat.Metadata that is a trivial
>>>>>>> extension of DefaultMetadata, but implements the HasColorTable interface
>>>>>>> * When your Writer is passed a SCIFIOConfig with an attached
>>>>>>> ColorTable, the framework will pass the ColorTable to the Writer's
>>>>>>> ImarisFormat.Metadata
>>>>>>> * In your Writer's writePlane implementation, you would check for a
>>>>>>> ColorTable on a) the provided Plane and b) the attached Metadata and write
>>>>>>> it out as appropriate.
>>>>>>>
>>>>>>> So asking the Plane for its ColorTable was the right thing to do.
>>>>>>>
>>>>>>
>>>>>> Okay, I'm watching the ticket, so once everything works as it should
>>>>>> I'll update this.
>>>>>>
>>>>>>
>>>>>>
>>>>>>> The fact that the ColorTable isn't being transferred from your
>>>>>>> original TIFF dataset is likely a bug. I'm not sure if it wasn't read
>>>>>>> properly on the TIFF side, or if it was read but didn't get passed to your
>>>>>>> ImarisFormat.Writer. Could you share your failing TIFF dataset so I can
>>>>>>> debug a bit more and figure out where it's going wrong? (note that you can
>>>>>>> use Fiji to upload a dataset[5] so that it won't be shared publicly)
>>>>>>>
>>>>>>>
>>>>>> My TIFF dataset was simply one of the ImageJ samples. I made it using
>>>>>> a pre-IJ2 version of FIJI by selecting File--Open samples--Mitosis and
>>>>>> saving as a TIFF. I then reopened it using SCIFIO.
>>>>>>
>>>>>> Best,
>>>>>> Henry
>>>>>>
>>>>>>
>>>>>>
>>>>>>>
>>>>>>> On Thu, Jan 15, 2015 at 11:07 AM, Henry Pinkard <
>>>>>>> henry.pinkard at gmail.com> wrote:
>>>>>>>
>>>>>>>> Hi Mark,
>>>>>>>>
>>>>>>>> Thanks for the quick reply!
>>>>>>>>
>>>>>>>>
>>>>>>>>> 1) Curtis knows this topic better than me, but I believe you can
>>>>>>>>> use the Maven NAR plugin[1] to build native code archives (.nar) which then
>>>>>>>>> can be distributed via a maven repository, such as our
>>>>>>>>> http://maven.imagej.net, and consumed as a normal Maven
>>>>>>>>> dependency by your writer. How is the native code you're using licensed?
>>>>>>>>>
>>>>>>>>
>>>>>>>> I don't actually need to build the native archives. They are
>>>>>>>> distributed as pre built binaries on the HDF website. Is it possible to
>>>>>>>> include these binaries directly in the maven repository? The license is a
>>>>>>>> "BSD style open-source license." More info can be found at:
>>>>>>>> http://www.hdfgroup.org/products/licenses.html
>>>>>>>>
>>>>>>>>
>>>>>>>> 2) You're absolutely right, AbstractWriter is provided to take care
>>>>>>>>> of as much boilerplate as possible, to try and keep what developers like
>>>>>>>>> you have to implement to a minimum. We avoid using public/protected fields
>>>>>>>>> in our libraries because of the problems they create, given that there is
>>>>>>>>> no way to respond when such fields are directly modified. The way to access
>>>>>>>>> these fields should be accessor/mutator methods (e.g. getMetadata(),
>>>>>>>>> isInitialized(...), etc...), with the expectation that they can be
>>>>>>>>> overridden if necessary. Since it feels like you're overriding too many of
>>>>>>>>> these, though, could you provide some more information about what
>>>>>>>>> functionality you needed to add/modify? If the API is lacking, my first
>>>>>>>>> choice would be to improve AbstractWriter so that it better meets your
>>>>>>>>> needs (and those of future developers!).
>>>>>>>>>
>>>>>>>>
>>>>>>>> As I look over this again, I think that this writer is probably an
>>>>>>>> exception that operates by different mechanisms than most Formats, and
>>>>>>>> probably doesn't warrant a change in the API. more on this below.
>>>>>>>>
>>>>>>>>
>>>>>>>>> 3) So, what you most likely want to do here is implement your own
>>>>>>>>> Metadata class for the ImarisFormat. DefaultMetadata was created as a dummy
>>>>>>>>> class to provide some minimal default implementation for formats that
>>>>>>>>> simply don't have any metadata to store (e.g. the JavaFormat, which only
>>>>>>>>> writes images). If you use DefaultMetadata, your Format should still work -
>>>>>>>>> but if you want to convert an ImarisFormat image to something else (such as
>>>>>>>>> an .ome.tif) you may be missing the opportunity to preserve any
>>>>>>>>> Imaris-specific metadata.
>>>>>>>>>
>>>>>>>>> I think this flow was not properly described in SCIFIO. To take a
>>>>>>>>> step towards improvement, I updated the custom format tutorial[2] to better
>>>>>>>>> explain the role of format-specific metadata.
>>>>>>>>>
>>>>>>>>> As to the actual LUT question, I suggest modeling the ImarisFormat
>>>>>>>>> after a simple format that has a LUT - such as BMPFormat[3]. You can see
>>>>>>>>> that the BMPFormat's metadata implements a HasColorTable interface[4]. It
>>>>>>>>> is then the role of the Parser to populate the metadata's color table[5].
>>>>>>>>> Then it is the role of the Reader to attach the color table to a plane[6].
>>>>>>>>>
>>>>>>>>> The SCIFIOConfig method you mentioned[7] is for external software
>>>>>>>>> to set a custom LUT when saving an image.
>>>>>>>>>
>>>>>>>>
>>>>>>>> Actually, I think the SCIFIO tutorials described the flow quite
>>>>>>>> well. The library that I'm adding in to the ImarisFormat is only for
>>>>>>>> writing IMS files and doesn't support reading them, so I opted to model
>>>>>>>> this off of JavaFormat. There are a few basic metadata fields needed for
>>>>>>>> writing, but I didn't think it necessary to create a format-specific
>>>>>>>> metadata class to hold them. Since Imaris is a software for only analysis,
>>>>>>>> no acquisition softwares save directly into IMS format. So the most
>>>>>>>> complete set of metadata will always reside in whatever format was used to
>>>>>>>> create the IMS file. If you think it would be better to create an
>>>>>>>> ImarisFormat.Metadata class, I'm happy to do so, but I don't think it would
>>>>>>>> affect the functionality of the format.
>>>>>>>>
>>>>>>>> I took a look at BMPFormat, and I think the color table here is
>>>>>>>> being read from a BMP file and then stored in the BMP specific metadata,
>>>>>>>> right? I'm actually looking to do the opposite--write an LUT into an IMS.
>>>>>>>> So then I should use the one provided by the SCIFIO config method, right?
>>>>>>>>
>>>>>>>> How about date/time stamps? Is there any way to extract this from
>>>>>>>> default metadata classes or does it need to be done in a metadata
>>>>>>>> format-specific manner?
>>>>>>>>
>>>>>>>> Best,
>>>>>>>> Henry
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> [1] https://github.com/maven-nar/nar-maven-plugin
>>>>>>>>> [2]
>>>>>>>>> https://github.com/scifio/scifio-tutorials/blob/5f9212a685c18ddc0cb4500e7ac4a3f1aad1940b/core/src/main/java/io/scif/tutorials/core/T3aCustomFormats.java#L123-133
>>>>>>>>> [3]
>>>>>>>>> https://github.com/scifio/scifio/blob/37822206e9ca842bb18596ef0f0312ad3f2e6d5f/src/main/java/io/scif/formats/BMPFormat.java
>>>>>>>>> [4]
>>>>>>>>> https://github.com/scifio/scifio/blob/37822206e9ca842bb18596ef0f0312ad3f2e6d5f/src/main/java/io/scif/formats/BMPFormat.java#L90-91
>>>>>>>>> [5]
>>>>>>>>> https://github.com/scifio/scifio/blob/37822206e9ca842bb18596ef0f0312ad3f2e6d5f/src/main/java/io/scif/formats/BMPFormat.java#L283-292
>>>>>>>>> [6]
>>>>>>>>> https://github.com/scifio/scifio/blob/37822206e9ca842bb18596ef0f0312ad3f2e6d5f/src/main/java/io/scif/formats/BMPFormat.java#L399-400
>>>>>>>>> [7]
>>>>>>>>> https://github.com/scifio/scifio/blob/37822206e9ca842bb18596ef0f0312ad3f2e6d5f/src/main/java/io/scif/config/SCIFIOConfig.java#L264-271
>>>>>>>>>
>>>>>>>>> On Wed, Jan 14, 2015 at 3:28 PM, Henry Pinkard <
>>>>>>>>> henry.pinkard at gmail.com> wrote:
>>>>>>>>>
>>>>>>>>>> Hi Curtis and Mark,
>>>>>>>>>>
>>>>>>>>>> Finally back to work on this project. I've created an
>>>>>>>>>> ImarisFormat class in my local copy of SCIFIO and have gotten a basic
>>>>>>>>>> version of ImarisFormat.Writer working. Several questions regarding this:
>>>>>>>>>>
>>>>>>>>>> 1) Since .ims files are a specific type of HDF5 file, I'm
>>>>>>>>>> utilizing the HDF Java library to do the writing. Maven is able to record a
>>>>>>>>>> dependency for the .jar file needed by this library, but I'm not sure how
>>>>>>>>>> to indicate a dependency on the the native files that the library uses
>>>>>>>>>> through the JNI. For testing purposes I just copied these libraries into
>>>>>>>>>> the SCIFIO root directory, but this is obviously not a solution. I assume
>>>>>>>>>> that there is an elegant way of doing this, since the BioFormats Imaris
>>>>>>>>>> reader is able to read these files. Any thoughts?
>>>>>>>>>>
>>>>>>>>>> 2) I'm unsure whether to have the writer extend AbstractWriter or
>>>>>>>>>> simply implement TypedWrtier. I would think that AbstractWriter is provided
>>>>>>>>>> as a stub to easily extend when making writer classes, but the private
>>>>>>>>>> access to all of this classes field's (metadata, initialized, etc.) mean
>>>>>>>>>> that I have to extend many more methods than I would like. Could these
>>>>>>>>>> fields be made protected, or should I switch to TypedWriter and have a lot
>>>>>>>>>> of empty, irrelevant methods?
>>>>>>>>>>
>>>>>>>>>> 3) I'm able to pull all the metadata required for writing these
>>>>>>>>>> files from DefaultMetadata and DefaultImageMetadata, with the exception of
>>>>>>>>>> two things. First, is there a way to get the acquisition date and
>>>>>>>>>> timestamps form individual planes? Second, the config.writerGetColorModel()
>>>>>>>>>> call that I copied from AbstractWriter returns null, as does
>>>>>>>>>> plane.getColorTable(). Is there some other way to get image lookup tables?
>>>>>>>>>> My test code opens a Tiff stack saved by FIJI that has LUTs applied, so I
>>>>>>>>>> would think they would be there. Is there another mechanism for getting
>>>>>>>>>> image LUTs?
>>>>>>>>>>
>>>>>>>>>> Excited to see this writer become part of SCIFIO in the near
>>>>>>>>>> future.
>>>>>>>>>>
>>>>>>>>>> Best,
>>>>>>>>>> Henry
>>>>>>>>>>
>>>>>>>>>> On Wed, Jul 16, 2014 at 4:32 PM, Curtis Rueden <ctrueden at wisc.edu
>>>>>>>>>> > wrote:
>>>>>>>>>>
>>>>>>>>>>> Hi Henry,
>>>>>>>>>>>
>>>>>>>>>>> > I've created a class in the io.scif.formats package for my
>>>>>>>>>>> writer.
>>>>>>>>>>> > Since I only have the writer components, but not the reader,
>>>>>>>>>>> how
>>>>>>>>>>> > should I go about implementing all of the component classes of
>>>>>>>>>>> Format
>>>>>>>>>>> > (Metadata, Parser, Checker, Reader, Writer, Translator). The
>>>>>>>>>>> first 4
>>>>>>>>>>> > are listed as mandatory in the tutorial, but it seems like I
>>>>>>>>>>> shouldn't
>>>>>>>>>>> > need all of them for writing functionality alone.
>>>>>>>>>>>
>>>>>>>>>>> Agreed, it should not be necessary to create Reader or Parser
>>>>>>>>>>> components. However, writer-only components are still a little rough around
>>>>>>>>>>> the edges. The relevant issue is:
>>>>>>>>>>>
>>>>>>>>>>> https://github.com/scifio/scifio/issues/211
>>>>>>>>>>>
>>>>>>>>>>> Feel free to comment on that so that GitHub sends you updates if
>>>>>>>>>>> you care about progress on it.
>>>>>>>>>>>
>>>>>>>>>>> But in the meantime, it should still be possible to create a
>>>>>>>>>>> writer-only component. For an example, see the JavaFormat:
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> https://github.com/scifio/scifio/blob/scifio-0.15.4/scifio/src/main/java/io/scif/formats/JavaFormat.java
>>>>>>>>>>>
>>>>>>>>>>> Regards,
>>>>>>>>>>> Curtis
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> On Wed, Jun 25, 2014 at 1:21 PM, Henry Pinkard <
>>>>>>>>>>> henry.pinkard at gmail.com> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> Hi Curtis,
>>>>>>>>>>>>
>>>>>>>>>>>> This all sounds great. I've cloned the SCIFIO core as well as
>>>>>>>>>>>> the tutorials, which have been quite helpful in getting things set up.
>>>>>>>>>>>>
>>>>>>>>>>>> I've created a class in the io.scif.formats package for my
>>>>>>>>>>>> writer. Since I only have the writer components, but not the reader, how
>>>>>>>>>>>> should I go about implementing all of the component classes of Format
>>>>>>>>>>>> (Metadata, Parser, Checker, Reader, Writer, Translator). The first 4 are
>>>>>>>>>>>> listed as mandatory in the tutorial, but it seems like I shouldn't need all
>>>>>>>>>>>> of them for writing functionality alone.
>>>>>>>>>>>>
>>>>>>>>>>>> Best,
>>>>>>>>>>>> Henry
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> On Sun, Jun 1, 2014 at 7:27 AM, Curtis Rueden <
>>>>>>>>>>>> ctrueden at wisc.edu> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> Hi Henry,
>>>>>>>>>>>>>
>>>>>>>>>>>>> > Over the past couple years, I've been developing and testing
>>>>>>>>>>>>> a java
>>>>>>>>>>>>> > library that is capable of writing Imaris .ims files. This
>>>>>>>>>>>>> library has
>>>>>>>>>>>>> > allowed me to build an ImageJ plugin that automatically
>>>>>>>>>>>>> stitches,
>>>>>>>>>>>>> > processes, and converts OME-TIFFs from Micro-Manager into
>>>>>>>>>>>>> Imaris
>>>>>>>>>>>>> > files, which in turn allows a significantly greater
>>>>>>>>>>>>> throughput of
>>>>>>>>>>>>> > imaging data with much less effort from users.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Sounds great!
>>>>>>>>>>>>>
>>>>>>>>>>>>> > This library has been instrumental to the workflow of users
>>>>>>>>>>>>> in the
>>>>>>>>>>>>> > imaging center in which I work, and I want to find a way to
>>>>>>>>>>>>> share its
>>>>>>>>>>>>> > utility with researchers everywhere. Incorporating it into
>>>>>>>>>>>>> the
>>>>>>>>>>>>> > Bio-Formats exporter would both increase its visibility and
>>>>>>>>>>>>> leverage
>>>>>>>>>>>>> > the multitude of formats on Bio-Formats' front end to make
>>>>>>>>>>>>> it usable
>>>>>>>>>>>>> > with all types of microscopy data.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Rather than implementing a Bio-Formats writer, I encourage you
>>>>>>>>>>>>> to check out SCIFIO [1, 2, 3]. Though still in beta, SCIFIO is the core I/O
>>>>>>>>>>>>> mechanism of ImageJ2, which is finally due for release later this week.
>>>>>>>>>>>>> SCIFIO uses the SciJava plugin framework, meaning your writer would be
>>>>>>>>>>>>> automatically discovered and used as appropriate with no additional work
>>>>>>>>>>>>> from you. And we have recently integrated SCIFIO directly into ImageJ at
>>>>>>>>>>>>> the core level, so things like File > Open now use it, including whatever
>>>>>>>>>>>>> format plugins are available. (SCIFIO also has a Bio-Formats format plugin,
>>>>>>>>>>>>> so that all of the BF formats work in ImageJ that way, too!)
>>>>>>>>>>>>>
>>>>>>>>>>>>> You could then serve your Imaris writer from its own ImageJ
>>>>>>>>>>>>> update site [4, 5], to make it available to all ImageJ users.
>>>>>>>>>>>>>
>>>>>>>>>>>>> > In addition, I've convinced Bitplane to make their format
>>>>>>>>>>>>> open source,
>>>>>>>>>>>>> > and I believe this may allow .ims files to grow beyond a
>>>>>>>>>>>>> proprietary
>>>>>>>>>>>>> > file format into a generalized multi-resolution format
>>>>>>>>>>>>> useful for a
>>>>>>>>>>>>> > variety of applications that deal with the visualization of
>>>>>>>>>>>>> extremely
>>>>>>>>>>>>> > large stitched images.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Glad to hear that Bitplane is willing to do this. In that
>>>>>>>>>>>>> case, if you do complete a SCIFIO Imaris writer and want to donate the code
>>>>>>>>>>>>> upstream, you could file a pull request against the SCIFIO LifeSci project
>>>>>>>>>>>>> [6] to contribute it there, since that project houses readers & writers for
>>>>>>>>>>>>> _open_ life sciences formats. So if Bitplane publishes an open
>>>>>>>>>>>>> specification, we would be willing to add it to the project.
>>>>>>>>>>>>>
>>>>>>>>>>>>> If you have any questions about SCIFIO, please feel free to
>>>>>>>>>>>>> use the SCIFIO mailing list [7]. Or if you go the Bio-Formats route, you
>>>>>>>>>>>>> can use ome-devel [8].
>>>>>>>>>>>>>
>>>>>>>>>>>>> Regards,
>>>>>>>>>>>>> Curtis
>>>>>>>>>>>>>
>>>>>>>>>>>>> [1] http://loci.wisc.edu/software/scifio
>>>>>>>>>>>>> [2] https://github.com/scifio/scifio
>>>>>>>>>>>>> [3] https://github.com/scifio/scifio-tutorials
>>>>>>>>>>>>> [4] http://wiki.imagej.net/Update_Sites
>>>>>>>>>>>>> [5] http://wiki.imagej.net/List_of_update_sites
>>>>>>>>>>>>>  [6] https://github.com/scifio/scifio-lifesci
>>>>>>>>>>>>> [7] http://scif.io/mailman/listinfo/scifio
>>>>>>>>>>>>> [8]
>>>>>>>>>>>>> http://lists.openmicroscopy.org.uk/mailman/listinfo/ome-devel/
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Thu, May 29, 2014 at 6:55 PM, Henry Pinkard <
>>>>>>>>>>>>> henry.pinkard at gmail.com> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> Hi Melissa and Curtis,
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Over the past couple years, I've been developing and testing
>>>>>>>>>>>>>> a java library that is capable of writing Imaris .ims files. This library
>>>>>>>>>>>>>> has allowed me to build an ImageJ plugin that automatically stitches,
>>>>>>>>>>>>>> processes, and converts OME-TIFFs from Micro-Manager into Imaris files,
>>>>>>>>>>>>>> which in turn allows a significantly greater throughput of imaging data
>>>>>>>>>>>>>> with much less effort from users.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> This library has been instrumental to the workflow of users
>>>>>>>>>>>>>> in the imaging center in which I work, and I want to find a way to share
>>>>>>>>>>>>>> its utility with researchers everywhere. Incorporating it into the
>>>>>>>>>>>>>> Bio-Formats exporter would both increase its visibility and leverage the
>>>>>>>>>>>>>> multitude of formats on Bio-Formats' front end to make it usable with all
>>>>>>>>>>>>>> types of microscopy data. In addition, I've convinced Bitplane to make
>>>>>>>>>>>>>> their format open source, and I believe this may allow .ims files to grow
>>>>>>>>>>>>>> beyond a proprietary file format into a generalized multi-resolution format
>>>>>>>>>>>>>> useful for a variety of applications that deal with the visualization of
>>>>>>>>>>>>>> extremely large stitched images. I'm happy to rework the library in
>>>>>>>>>>>>>> whatever ways make it easiest to incorporate on your end. Let me know your
>>>>>>>>>>>>>> thoughts on how to best proceed.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Best,
>>>>>>>>>>>>>> Henry
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> _______________________________________________
>>>>>>>>>> SCIFIO mailing list
>>>>>>>>>> SCIFIO at scif.io
>>>>>>>>>> http://scif.io/mailman/listinfo/scifio
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>> On Thu, Jan 15, 2015 at 4:21 PM, Mark Hiner <hiner at wisc.edu> wrote:
>>>>>>
>>>>>>> Hi Henry,
>>>>>>>
>>>>>>> > Is it possible to include these binaries directly in the maven
>>>>>>> repository?
>>>>>>>
>>>>>>> Not directly. But since their license allows them to be
>>>>>>> redistributed, I think what we want is to make NARs out of them[1] and put
>>>>>>> the NAR in the maven repository.
>>>>>>>
>>>>>>> However, there are almost certainly limitations right now. I
>>>>>>> personally do not have experience with the nar-maven-plugin, but I would go
>>>>>>> into it expecting some hurdles.
>>>>>>>
>>>>>>> On the other hand, I think your ImarisFormat is an awesome use case
>>>>>>> for the nar-maven-plugin and this is a great opportunity to work out
>>>>>>> problems and document NAR usage in SCIFIO/ImageJ2/Fiji! So I'd be happy to
>>>>>>> help investigate further.
>>>>>>>
>>>>>>> Also, it's worth noting the ImageJ wiki also has a higher-level
>>>>>>> discussion of using native code[2], though it is outdated (and doesn't
>>>>>>> discuss the nar-maven-plugin).
>>>>>>>
>>>>>>> >Actually, I think the SCIFIO tutorials described the flow quite
>>>>>>> well.
>>>>>>>
>>>>>>> Thanks for that! I still felt the need to revise the format
>>>>>>> tutorial[3], as there were some basic elements that simply weren't
>>>>>>> discussed (like ColorTables).
>>>>>>>
>>>>>>> >The library that I'm adding in to the ImarisFormat is only for
>>>>>>> writing IMS files and doesn't support reading them, so I opted to model
>>>>>>> this off of JavaFormat.
>>>>>>> >There are a few basic metadata fields needed for writing, but I
>>>>>>> didn't think it necessary to create a format-specific metadata class to
>>>>>>> hold them.
>>>>>>> >Since Imaris is a software for only analysis, no acquisition
>>>>>>> softwares save directly into IMS format.
>>>>>>> >So the most complete set of metadata will always reside in whatever
>>>>>>> format was used to create the IMS file.
>>>>>>>
>>>>>>> This makes a lot of sense - thank you for the explanation. I now
>>>>>>> think your decision not to have an ImarisFormat.Metadata class is correct -
>>>>>>> except for the fact that ImarisFormat has LUTs (see below).
>>>>>>>
>>>>>>> >I took a look at BMPFormat, and I think the color table here is
>>>>>>> being read from a BMP file and then stored in the BMP specific metadata,
>>>>>>> right? I'm actually looking to do
>>>>>>> >the opposite--write an LUT into an IMS. So then I should use the
>>>>>>> one provided by the SCIFIO config method, right?
>>>>>>>
>>>>>>> Sort of. I think the SCIFIOConfig and Writer.getColorModel API are
>>>>>>> incorrect to use java.awt.image.ColorModels. I opened a ticket[4] to update
>>>>>>> them to the ImgLib2 ColorTable class. When that ticket is complete, this is
>>>>>>> how everything should work:
>>>>>>>
>>>>>>> * You should make an ImarisFormat.Metadata that is a trivial
>>>>>>> extension of DefaultMetadata, but implements the HasColorTable interface
>>>>>>> * When your Writer is passed a SCIFIOConfig with an attached
>>>>>>> ColorTable, the framework will pass the ColorTable to the Writer's
>>>>>>> ImarisFormat.Metadata
>>>>>>> * In your Writer's writePlane implementation, you would check for a
>>>>>>> ColorTable on a) the provided Plane and b) the attached Metadata and write
>>>>>>> it out as appropriate.
>>>>>>>
>>>>>>> So asking the Plane for its ColorTable was the right thing to do.
>>>>>>>
>>>>>>> The fact that the ColorTable isn't being transferred from your
>>>>>>> original TIFF dataset is likely a bug. I'm not sure if it wasn't read
>>>>>>> properly on the TIFF side, or if it was read but didn't get passed to your
>>>>>>> ImarisFormat.Writer. Could you share your failing TIFF dataset so I can
>>>>>>> debug a bit more and figure out where it's going wrong? (note that you can
>>>>>>> use Fiji to upload a dataset[5] so that it won't be shared publicly)
>>>>>>>
>>>>>>> Looking forward to getting this working for you - thanks again!
>>>>>>>
>>>>>>> Best,
>>>>>>> Mark
>>>>>>>
>>>>>>> [1] http://maven-nar.github.io/index.html
>>>>>>> [2] http://imagej.net/Native
>>>>>>> [3]
>>>>>>> https://github.com/scifio/scifio-tutorials/commit/5780247e65b0671f1b0e4bba744bde7661114531
>>>>>>> [4] https://github.com/scifio/scifio/issues/239
>>>>>>> [5] http://imagej.net/Upload_Sample_Image
>>>>>>>
>>>>>>> On Thu, Jan 15, 2015 at 11:07 AM, Henry Pinkard <
>>>>>>> henry.pinkard at gmail.com> wrote:
>>>>>>>
>>>>>>>> Hi Mark,
>>>>>>>>
>>>>>>>> Thanks for the quick reply!
>>>>>>>>
>>>>>>>>
>>>>>>>>> 1) Curtis knows this topic better than me, but I believe you can
>>>>>>>>> use the Maven NAR plugin[1] to build native code archives (.nar) which then
>>>>>>>>> can be distributed via a maven repository, such as our
>>>>>>>>> http://maven.imagej.net, and consumed as a normal Maven
>>>>>>>>> dependency by your writer. How is the native code you're using licensed?
>>>>>>>>>
>>>>>>>>
>>>>>>>> I don't actually need to build the native archives. They are
>>>>>>>> distributed as pre built binaries on the HDF website. Is it possible to
>>>>>>>> include these binaries directly in the maven repository? The license is a
>>>>>>>> "BSD style open-source license." More info can be found at:
>>>>>>>> http://www.hdfgroup.org/products/licenses.html
>>>>>>>>
>>>>>>>>
>>>>>>>> 2) You're absolutely right, AbstractWriter is provided to take care
>>>>>>>>> of as much boilerplate as possible, to try and keep what developers like
>>>>>>>>> you have to implement to a minimum. We avoid using public/protected fields
>>>>>>>>> in our libraries because of the problems they create, given that there is
>>>>>>>>> no way to respond when such fields are directly modified. The way to access
>>>>>>>>> these fields should be accessor/mutator methods (e.g. getMetadata(),
>>>>>>>>> isInitialized(...), etc...), with the expectation that they can be
>>>>>>>>> overridden if necessary. Since it feels like you're overriding too many of
>>>>>>>>> these, though, could you provide some more information about what
>>>>>>>>> functionality you needed to add/modify? If the API is lacking, my first
>>>>>>>>> choice would be to improve AbstractWriter so that it better meets your
>>>>>>>>> needs (and those of future developers!).
>>>>>>>>>
>>>>>>>>
>>>>>>>> As I look over this again, I think that this writer is probably an
>>>>>>>> exception that operates by different mechanisms than most Formats, and
>>>>>>>> probably doesn't warrant a change in the API. more on this below.
>>>>>>>>
>>>>>>>>
>>>>>>>>> 3) So, what you most likely want to do here is implement your own
>>>>>>>>> Metadata class for the ImarisFormat. DefaultMetadata was created as a dummy
>>>>>>>>> class to provide some minimal default implementation for formats that
>>>>>>>>> simply don't have any metadata to store (e.g. the JavaFormat, which only
>>>>>>>>> writes images). If you use DefaultMetadata, your Format should still work -
>>>>>>>>> but if you want to convert an ImarisFormat image to something else (such as
>>>>>>>>> an .ome.tif) you may be missing the opportunity to preserve any
>>>>>>>>> Imaris-specific metadata.
>>>>>>>>>
>>>>>>>>> I think this flow was not properly described in SCIFIO. To take a
>>>>>>>>> step towards improvement, I updated the custom format tutorial[2] to better
>>>>>>>>> explain the role of format-specific metadata.
>>>>>>>>>
>>>>>>>>> As to the actual LUT question, I suggest modeling the ImarisFormat
>>>>>>>>> after a simple format that has a LUT - such as BMPFormat[3]. You can see
>>>>>>>>> that the BMPFormat's metadata implements a HasColorTable interface[4]. It
>>>>>>>>> is then the role of the Parser to populate the metadata's color table[5].
>>>>>>>>> Then it is the role of the Reader to attach the color table to a plane[6].
>>>>>>>>>
>>>>>>>>> The SCIFIOConfig method you mentioned[7] is for external software
>>>>>>>>> to set a custom LUT when saving an image.
>>>>>>>>>
>>>>>>>>
>>>>>>>> Actually, I think the SCIFIO tutorials described the flow quite
>>>>>>>> well. The library that I'm adding in to the ImarisFormat is only for
>>>>>>>> writing IMS files and doesn't support reading them, so I opted to model
>>>>>>>> this off of JavaFormat. There are a few basic metadata fields needed for
>>>>>>>> writing, but I didn't think it necessary to create a format-specific
>>>>>>>> metadata class to hold them. Since Imaris is a software for only analysis,
>>>>>>>> no acquisition softwares save directly into IMS format. So the most
>>>>>>>> complete set of metadata will always reside in whatever format was used to
>>>>>>>> create the IMS file. If you think it would be better to create an
>>>>>>>> ImarisFormat.Metadata class, I'm happy to do so, but I don't think it would
>>>>>>>> affect the functionality of the format.
>>>>>>>>
>>>>>>>> I took a look at BMPFormat, and I think the color table here is
>>>>>>>> being read from a BMP file and then stored in the BMP specific metadata,
>>>>>>>> right? I'm actually looking to do the opposite--write an LUT into an IMS.
>>>>>>>> So then I should use the one provided by the SCIFIO config method, right?
>>>>>>>>
>>>>>>>> How about date/time stamps? Is there any way to extract this from
>>>>>>>> default metadata classes or does it need to be done in a metadata
>>>>>>>> format-specific manner?
>>>>>>>>
>>>>>>>> Best,
>>>>>>>> Henry
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> [1] https://github.com/maven-nar/nar-maven-plugin
>>>>>>>>> [2]
>>>>>>>>> https://github.com/scifio/scifio-tutorials/blob/5f9212a685c18ddc0cb4500e7ac4a3f1aad1940b/core/src/main/java/io/scif/tutorials/core/T3aCustomFormats.java#L123-133
>>>>>>>>> [3]
>>>>>>>>> https://github.com/scifio/scifio/blob/37822206e9ca842bb18596ef0f0312ad3f2e6d5f/src/main/java/io/scif/formats/BMPFormat.java
>>>>>>>>> [4]
>>>>>>>>> https://github.com/scifio/scifio/blob/37822206e9ca842bb18596ef0f0312ad3f2e6d5f/src/main/java/io/scif/formats/BMPFormat.java#L90-91
>>>>>>>>> [5]
>>>>>>>>> https://github.com/scifio/scifio/blob/37822206e9ca842bb18596ef0f0312ad3f2e6d5f/src/main/java/io/scif/formats/BMPFormat.java#L283-292
>>>>>>>>> [6]
>>>>>>>>> https://github.com/scifio/scifio/blob/37822206e9ca842bb18596ef0f0312ad3f2e6d5f/src/main/java/io/scif/formats/BMPFormat.java#L399-400
>>>>>>>>> [7]
>>>>>>>>> https://github.com/scifio/scifio/blob/37822206e9ca842bb18596ef0f0312ad3f2e6d5f/src/main/java/io/scif/config/SCIFIOConfig.java#L264-271
>>>>>>>>>
>>>>>>>>> On Wed, Jan 14, 2015 at 3:28 PM, Henry Pinkard <
>>>>>>>>> henry.pinkard at gmail.com> wrote:
>>>>>>>>>
>>>>>>>>>> Hi Curtis and Mark,
>>>>>>>>>>
>>>>>>>>>> Finally back to work on this project. I've created an
>>>>>>>>>> ImarisFormat class in my local copy of SCIFIO and have gotten a basic
>>>>>>>>>> version of ImarisFormat.Writer working. Several questions regarding this:
>>>>>>>>>>
>>>>>>>>>> 1) Since .ims files are a specific type of HDF5 file, I'm
>>>>>>>>>> utilizing the HDF Java library to do the writing. Maven is able to record a
>>>>>>>>>> dependency for the .jar file needed by this library, but I'm not sure how
>>>>>>>>>> to indicate a dependency on the the native files that the library uses
>>>>>>>>>> through the JNI. For testing purposes I just copied these libraries into
>>>>>>>>>> the SCIFIO root directory, but this is obviously not a solution. I assume
>>>>>>>>>> that there is an elegant way of doing this, since the BioFormats Imaris
>>>>>>>>>> reader is able to read these files. Any thoughts?
>>>>>>>>>>
>>>>>>>>>> 2) I'm unsure whether to have the writer extend AbstractWriter or
>>>>>>>>>> simply implement TypedWrtier. I would think that AbstractWriter is provided
>>>>>>>>>> as a stub to easily extend when making writer classes, but the private
>>>>>>>>>> access to all of this classes field's (metadata, initialized, etc.) mean
>>>>>>>>>> that I have to extend many more methods than I would like. Could these
>>>>>>>>>> fields be made protected, or should I switch to TypedWriter and have a lot
>>>>>>>>>> of empty, irrelevant methods?
>>>>>>>>>>
>>>>>>>>>> 3) I'm able to pull all the metadata required for writing these
>>>>>>>>>> files from DefaultMetadata and DefaultImageMetadata, with the exception of
>>>>>>>>>> two things. First, is there a way to get the acquisition date and
>>>>>>>>>> timestamps form individual planes? Second, the config.writerGetColorModel()
>>>>>>>>>> call that I copied from AbstractWriter returns null, as does
>>>>>>>>>> plane.getColorTable(). Is there some other way to get image lookup tables?
>>>>>>>>>> My test code opens a Tiff stack saved by FIJI that has LUTs applied, so I
>>>>>>>>>> would think they would be there. Is there another mechanism for getting
>>>>>>>>>> image LUTs?
>>>>>>>>>>
>>>>>>>>>> Excited to see this writer become part of SCIFIO in the near
>>>>>>>>>> future.
>>>>>>>>>>
>>>>>>>>>> Best,
>>>>>>>>>> Henry
>>>>>>>>>>
>>>>>>>>>> On Wed, Jul 16, 2014 at 4:32 PM, Curtis Rueden <ctrueden at wisc.edu
>>>>>>>>>> > wrote:
>>>>>>>>>>
>>>>>>>>>>> Hi Henry,
>>>>>>>>>>>
>>>>>>>>>>> > I've created a class in the io.scif.formats package for my
>>>>>>>>>>> writer.
>>>>>>>>>>> > Since I only have the writer components, but not the reader,
>>>>>>>>>>> how
>>>>>>>>>>> > should I go about implementing all of the component classes of
>>>>>>>>>>> Format
>>>>>>>>>>> > (Metadata, Parser, Checker, Reader, Writer, Translator). The
>>>>>>>>>>> first 4
>>>>>>>>>>> > are listed as mandatory in the tutorial, but it seems like I
>>>>>>>>>>> shouldn't
>>>>>>>>>>> > need all of them for writing functionality alone.
>>>>>>>>>>>
>>>>>>>>>>> Agreed, it should not be necessary to create Reader or Parser
>>>>>>>>>>> components. However, writer-only components are still a little rough around
>>>>>>>>>>> the edges. The relevant issue is:
>>>>>>>>>>>
>>>>>>>>>>> https://github.com/scifio/scifio/issues/211
>>>>>>>>>>>
>>>>>>>>>>> Feel free to comment on that so that GitHub sends you updates if
>>>>>>>>>>> you care about progress on it.
>>>>>>>>>>>
>>>>>>>>>>> But in the meantime, it should still be possible to create a
>>>>>>>>>>> writer-only component. For an example, see the JavaFormat:
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> https://github.com/scifio/scifio/blob/scifio-0.15.4/scifio/src/main/java/io/scif/formats/JavaFormat.java
>>>>>>>>>>>
>>>>>>>>>>> Regards,
>>>>>>>>>>> Curtis
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> On Wed, Jun 25, 2014 at 1:21 PM, Henry Pinkard <
>>>>>>>>>>> henry.pinkard at gmail.com> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> Hi Curtis,
>>>>>>>>>>>>
>>>>>>>>>>>> This all sounds great. I've cloned the SCIFIO core as well as
>>>>>>>>>>>> the tutorials, which have been quite helpful in getting things set up.
>>>>>>>>>>>>
>>>>>>>>>>>> I've created a class in the io.scif.formats package for my
>>>>>>>>>>>> writer. Since I only have the writer components, but not the reader, how
>>>>>>>>>>>> should I go about implementing all of the component classes of Format
>>>>>>>>>>>> (Metadata, Parser, Checker, Reader, Writer, Translator). The first 4 are
>>>>>>>>>>>> listed as mandatory in the tutorial, but it seems like I shouldn't need all
>>>>>>>>>>>> of them for writing functionality alone.
>>>>>>>>>>>>
>>>>>>>>>>>> Best,
>>>>>>>>>>>> Henry
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> On Sun, Jun 1, 2014 at 7:27 AM, Curtis Rueden <
>>>>>>>>>>>> ctrueden at wisc.edu> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> Hi Henry,
>>>>>>>>>>>>>
>>>>>>>>>>>>> > Over the past couple years, I've been developing and testing
>>>>>>>>>>>>> a java
>>>>>>>>>>>>> > library that is capable of writing Imaris .ims files. This
>>>>>>>>>>>>> library has
>>>>>>>>>>>>> > allowed me to build an ImageJ plugin that automatically
>>>>>>>>>>>>> stitches,
>>>>>>>>>>>>> > processes, and converts OME-TIFFs from Micro-Manager into
>>>>>>>>>>>>> Imaris
>>>>>>>>>>>>> > files, which in turn allows a significantly greater
>>>>>>>>>>>>> throughput of
>>>>>>>>>>>>> > imaging data with much less effort from users.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Sounds great!
>>>>>>>>>>>>>
>>>>>>>>>>>>> > This library has been instrumental to the workflow of users
>>>>>>>>>>>>> in the
>>>>>>>>>>>>> > imaging center in which I work, and I want to find a way to
>>>>>>>>>>>>> share its
>>>>>>>>>>>>> > utility with researchers everywhere. Incorporating it into
>>>>>>>>>>>>> the
>>>>>>>>>>>>> > Bio-Formats exporter would both increase its visibility and
>>>>>>>>>>>>> leverage
>>>>>>>>>>>>> > the multitude of formats on Bio-Formats' front end to make
>>>>>>>>>>>>> it usable
>>>>>>>>>>>>> > with all types of microscopy data.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Rather than implementing a Bio-Formats writer, I encourage you
>>>>>>>>>>>>> to check out SCIFIO [1, 2, 3]. Though still in beta, SCIFIO is the core I/O
>>>>>>>>>>>>> mechanism of ImageJ2, which is finally due for release later this week.
>>>>>>>>>>>>> SCIFIO uses the SciJava plugin framework, meaning your writer would be
>>>>>>>>>>>>> automatically discovered and used as appropriate with no additional work
>>>>>>>>>>>>> from you. And we have recently integrated SCIFIO directly into ImageJ at
>>>>>>>>>>>>> the core level, so things like File > Open now use it, including whatever
>>>>>>>>>>>>> format plugins are available. (SCIFIO also has a Bio-Formats format plugin,
>>>>>>>>>>>>> so that all of the BF formats work in ImageJ that way, too!)
>>>>>>>>>>>>>
>>>>>>>>>>>>> You could then serve your Imaris writer from its own ImageJ
>>>>>>>>>>>>> update site [4, 5], to make it available to all ImageJ users.
>>>>>>>>>>>>>
>>>>>>>>>>>>> > In addition, I've convinced Bitplane to make their format
>>>>>>>>>>>>> open source,
>>>>>>>>>>>>> > and I believe this may allow .ims files to grow beyond a
>>>>>>>>>>>>> proprietary
>>>>>>>>>>>>> > file format into a generalized multi-resolution format
>>>>>>>>>>>>> useful for a
>>>>>>>>>>>>> > variety of applications that deal with the visualization of
>>>>>>>>>>>>> extremely
>>>>>>>>>>>>> > large stitched images.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Glad to hear that Bitplane is willing to do this. In that
>>>>>>>>>>>>> case, if you do complete a SCIFIO Imaris writer and want to donate the code
>>>>>>>>>>>>> upstream, you could file a pull request against the SCIFIO LifeSci project
>>>>>>>>>>>>> [6] to contribute it there, since that project houses readers & writers for
>>>>>>>>>>>>> _open_ life sciences formats. So if Bitplane publishes an open
>>>>>>>>>>>>> specification, we would be willing to add it to the project.
>>>>>>>>>>>>>
>>>>>>>>>>>>> If you have any questions about SCIFIO, please feel free to
>>>>>>>>>>>>> use the SCIFIO mailing list [7]. Or if you go the Bio-Formats route, you
>>>>>>>>>>>>> can use ome-devel [8].
>>>>>>>>>>>>>
>>>>>>>>>>>>> Regards,
>>>>>>>>>>>>> Curtis
>>>>>>>>>>>>>
>>>>>>>>>>>>> [1] http://loci.wisc.edu/software/scifio
>>>>>>>>>>>>> [2] https://github.com/scifio/scifio
>>>>>>>>>>>>> [3] https://github.com/scifio/scifio-tutorials
>>>>>>>>>>>>> [4] http://wiki.imagej.net/Update_Sites
>>>>>>>>>>>>> [5] http://wiki.imagej.net/List_of_update_sites
>>>>>>>>>>>>>  [6] https://github.com/scifio/scifio-lifesci
>>>>>>>>>>>>> [7] http://scif.io/mailman/listinfo/scifio
>>>>>>>>>>>>> [8]
>>>>>>>>>>>>> http://lists.openmicroscopy.org.uk/mailman/listinfo/ome-devel/
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Thu, May 29, 2014 at 6:55 PM, Henry Pinkard <
>>>>>>>>>>>>> henry.pinkard at gmail.com> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> Hi Melissa and Curtis,
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Over the past couple years, I've been developing and testing
>>>>>>>>>>>>>> a java library that is capable of writing Imaris .ims files. This library
>>>>>>>>>>>>>> has allowed me to build an ImageJ plugin that automatically stitches,
>>>>>>>>>>>>>> processes, and converts OME-TIFFs from Micro-Manager into Imaris files,
>>>>>>>>>>>>>> which in turn allows a significantly greater throughput of imaging data
>>>>>>>>>>>>>> with much less effort from users.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> This library has been instrumental to the workflow of users
>>>>>>>>>>>>>> in the imaging center in which I work, and I want to find a way to share
>>>>>>>>>>>>>> its utility with researchers everywhere. Incorporating it into the
>>>>>>>>>>>>>> Bio-Formats exporter would both increase its visibility and leverage the
>>>>>>>>>>>>>> multitude of formats on Bio-Formats' front end to make it usable with all
>>>>>>>>>>>>>> types of microscopy data. In addition, I've convinced Bitplane to make
>>>>>>>>>>>>>> their format open source, and I believe this may allow .ims files to grow
>>>>>>>>>>>>>> beyond a proprietary file format into a generalized multi-resolution format
>>>>>>>>>>>>>> useful for a variety of applications that deal with the visualization of
>>>>>>>>>>>>>> extremely large stitched images. I'm happy to rework the library in
>>>>>>>>>>>>>> whatever ways make it easiest to incorporate on your end. Let me know your
>>>>>>>>>>>>>> thoughts on how to best proceed.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Best,
>>>>>>>>>>>>>> Henry
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> _______________________________________________
>>>>>>>>>> SCIFIO mailing list
>>>>>>>>>> SCIFIO at scif.io
>>>>>>>>>> http://scif.io/mailman/listinfo/scifio
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://scif.io/pipermail/scifio/attachments/20150126/3aa56207/attachment-0002.html>