Issue 59041: HDF5 HaloFinder preloading

Keyboard Shortcuts

	File
u :	up to issue
m :	publish + mail comments
M :	edit review message
j / k :	jump to file after / before current file
J / K :	jump to next file with a comment after / before current file
	Side-by-side diff
i :	toggle intra-line diffs
e :	expand all comments
c :	collapse all comments
s :	toggle showing all comments
n / p :	next / previous diff chunk or comment
N / P :	next / previous comment
<Up> / <Down> :	next / previous line
<Enter> :	respond to / edit current comment
d :	mark current comment as done

	Issue
u :	up to list of issues
m :	publish + mail comments
j / k :	jump to patch after / before current patch
o / <Enter> :	open current patch in side-by-side view
i :	open current patch in unified diff view

	Issue List
j / k :	jump to issue after / before current issue
o / <Enter> :	open current issue
# :	close issue

	Comment/message editing
<Ctrl> + s or <Ctrl> + Enter :	save comment
<Esc> :	cancel edit

Issue 59041: HDF5 HaloFinder preloading

Can't Edit
Can't Publish+Mail
Start Review

Created:
15 years, 10 months ago by stephenskory

Modified:
1 year, 4 months ago

Reviewers:
matthewturk

CC:
yt-dev_lists.spacepope.org

Base URL:
http://svn.enzotools.org/yt/trunk/yt/lagos/

Visibility:
Public.

Patch Set 1 #

Total comments: 3

Patch Set 2 : DerivedQuantities attempt #

Created: 15 years, 10 months ago

Download [raw] [tar.bz2]

	Unified diffs	Side-by-side diffs	Delta from patch set	Stats (+21 lines, -12 lines)			Patch
M	HaloFinding.py	View	1	7 chunks	+18 lines, -9 lines	0 comments	Download
M	hop/hop_hop.c	View	1	1 chunk	+1 line, -1 line	0 comments	Download
M	hop/hop_regroup.c	View	1	1 chunk	+2 lines, -2 lines	0 comments	Download

Messages

Total messages: 1

Expand All Messages | Collapse All Messages

matthewturk

15 years, 10 months ago (2009-05-05 16:32:29 UTC) #1

Looks mostly okay, but I think we need to rethink the initial mass calculation
before it gets committed.

http://codereview.appspot.com/59041/diff/1/2
File HaloFinding.py (right):

http://codereview.appspot.com/59041/diff/1/2#newcode197
Line 197: 
Not sure we need this?

http://codereview.appspot.com/59041/diff/1/2#newcode433
Line 433: padded, LE, RE, self._data_source =
self._partition_hierarchy_3d(padding=self.padding)
We should get rid of this initial step of partitioning the entire hierarchy, and
instead move to using a DerivedQuantity.  I'd say something like

all_data = self.pf.h.all_data()
all_data.quantities["TotalQuantity"]("ParticleMassMsun", lazy_reader=True)

which will automatically parallelize.  This would get rid of most of the
following lines, and we could avoid the mpi_allsum, too.  I'll need to add in
the preloading to the DQ object, however.

http://codereview.appspot.com/59041/diff/1/2#newcode450
Line 450: self._data_source.hierarchy.queue)
Should be fine.

Expand All Messages | Collapse All Messages