LinuxQuestions.org
Download your favorite Linux distribution at LQ ISO.
Go Back   LinuxQuestions.org > Forums > Non-*NIX Forums > Programming
User Name
Password
Programming This forum is for all programming questions.
The question does not have to be directly related to Linux and any language is fair game.

Notices


Reply
  Search this Thread
Old 06-09-2017, 03:09 PM   #1
wagscat123
Member
 
Registered: Jan 2009
Location: Maryland-Pennsylvania border, USA
Distribution: openSUSE 42.2, 42.3, Tumbleweed, OS X 10.10, Ubuntu 17.04, and Linux Mint 17.3/18.1
Posts: 659
Blog Entries: 9

Rep: Reputation: 88
Importing H5 dataset into Python


Hi all,

I've got an H5 dataset, and I've been able to get it into Panoply. I opened the H5 file in python, and used h5py's create_dataset to create a dataset with the shape option set to "1024,".

However, printing any element of the dataset returns a value of 0. How do I find the data that corresponds to what Panoply finds in the attached screenshot?

Many thanks for any pointers! I'm not as fluent in Python yet as I am in IDL.
Attached Thumbnails
Click image for larger version

Name:	panoply.png
Views:	20
Size:	142.4 KB
ID:	25225  
 
Old 06-09-2017, 08:25 PM   #2
norobro
Member
 
Registered: Feb 2006
Distribution: Debian Sid
Posts: 666

Rep: Reputation: 253Reputation: 253Reputation: 253
Caveat: I know nothing about this, but hopefully the following will help.

I don't think that you have to create anything, just load your file and find the dataset that you want.

I took a look at the h5py tutorial: http://docs.h5py.org/en/latest/high/file.html
And downloaded an h5 file from here: https://support.hdfgroup.org/ftp/HDF...iles/tutorial/
Code:
$ ipython3

In [1]: import h5py

In [2]: fp = h5py.File('SVM01.h5', 'r')

In [3]: list(fp.keys())
Out[3]: ['All_Data', 'Data_Products']

In [4]: list(fp['All_Data'].keys())
Out[4]: ['VIIRS-M1-SDR_All']

In [5]: list(fp['All_Data/VIIRS-M1-SDR_All'].keys())
Out[5]: 
['ModeGran',
 'ModeScan',
 'NumberOfBadChecksums',
 'NumberOfDiscardedPkts',
 'NumberOfMissingPkts',
 'NumberOfScans',
 'PadByte1',
 'QF1_VIIRSMBANDSDR',
 'QF2_SCAN_SDR',
 'QF3_SCAN_RDR',
 'QF4_SCAN_SDR',
 'QF5_GRAN_BADDETECTOR',
 'Radiance',
 'RadianceFactors',
 'Reflectance',
 'ReflectanceFactors']

In [7]: data = fp['All_Data/VIIRS-M1-SDR_All/Radiance']

In [8]: print(len(data))
768

In [9]: print(data[42])
[20905 20716 20038 ..., 23681 23699 23699]

In [10]: type(data[42])
Out[10]: numpy.ndarray

In [11]: len(data[42])
Out[11]: 3200
 
1 members found this post helpful.
Old 06-10-2017, 11:26 AM   #3
wagscat123
Member
 
Registered: Jan 2009
Location: Maryland-Pennsylvania border, USA
Distribution: openSUSE 42.2, 42.3, Tumbleweed, OS X 10.10, Ubuntu 17.04, and Linux Mint 17.3/18.1
Posts: 659
Blog Entries: 9

Original Poster
Rep: Reputation: 88
I already have been using those guides, but thanks for the response.

I'm still open though if anyone has any pointers.
 
Old 06-11-2017, 01:15 PM   #4
John VV
LQ Muse
 
Registered: Aug 2005
Location: A2 area Mi.
Posts: 17,169

Rep: Reputation: 2493Reputation: 2493Reputation: 2493Reputation: 2493Reputation: 2493Reputation: 2493Reputation: 2493Reputation: 2493Reputation: 2493Reputation: 2493Reputation: 2493
for hdf5 datasets i normally use gdal or qgis


there is a converter to ascii tables -- and it works great

" HDFView "
https://support.hdfgroup.org/products/java/hdfview/

http://img260.imagevenue.com/img.php..._122_224lo.jpg

it uses a spreadsheet layout but can export to a raw CSV file that gmic or other editors can use
http://img251.imagevenue.com/img.php..._122_246lo.jpg
 
1 members found this post helpful.
Old 06-11-2017, 09:20 PM   #5
wagscat123
Member
 
Registered: Jan 2009
Location: Maryland-Pennsylvania border, USA
Distribution: openSUSE 42.2, 42.3, Tumbleweed, OS X 10.10, Ubuntu 17.04, and Linux Mint 17.3/18.1
Posts: 659
Blog Entries: 9

Original Poster
Rep: Reputation: 88
Many thanks for both of your responses.

I think I've managed to crack into the data files with norobro's example, but now the trickier part is doing the analysis. I'll keep this thread open in case I end up with a new question about doing H5 stuff. The HDFView program helped me see what I wanted.
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
[SOLVED] Python and importing Ztcoracat Programming 11 06-21-2015 03:46 PM
Python -- Importing Modules Eahil Programming 3 06-17-2013 02:30 PM
[SOLVED] [Python] Importing and using tkinter Mr. Alex Programming 3 09-22-2012 08:43 AM
Importing python script nishitmukesh Linux - Newbie 4 04-11-2011 01:56 AM
importing trouble....python...regarding Tkinter maseby Programming 0 11-03-2003 01:54 PM

LinuxQuestions.org > Forums > Non-*NIX Forums > Programming

All times are GMT -5. The time now is 11:49 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration