Issue 200130043: Added detection and support for ParaVirtualized XEN guests. (Closed)
|
Created:
9 years, 2 months ago by Jordi Modified:
9 years, 2 months ago CC:
rekall-dev_googlegroups.com Base URL:
https://github.com/google/rekall@master Visibility:
Public. |
DescriptionAdded detection and support for ParaVirtualized XEN guests.
Refactored find_dtb plugins to do os-specific checks in the os-specific plugins instead of the base class.
BUG=
R=scudette@gmail.com
Committed: 366bc97
Patch Set 1 #
Total comments: 1
Patch Set 2 : Added better docstrings #Patch Set 3 : #
Total comments: 10
Patch Set 4 : Made os-specific address space selection in find_dtb be in the os-specific find_dtb plugins. #Patch Set 5 : First round of fixes. #
MessagesTotal messages: 15
https://codereview.appspot.com/200130043/diff/1/rekall/plugins/linux/common.py File rekall/plugins/linux/common.py (right): https://codereview.appspot.com/200130043/diff/1/rekall/plugins/linux/common.p... rekall/plugins/linux/common.py:138: phys_startup_64 = (profile.get_constant("phys_startup_64", False) or This is in preparation for using /proc/kallsyms instead of the System.map when unavailable. kallsyms doesn't export phys_startup_64 but it can be calculated.
Sign in to reply to this message.
https://codereview.appspot.com/200130043/diff/40001/rekall/plugins/addrspaces... File rekall/plugins/addrspaces/amd64.py (right): https://codereview.appspot.com/200130043/diff/40001/rekall/plugins/addrspaces... rekall/plugins/addrspaces/amd64.py:316: docstring https://codereview.appspot.com/200130043/diff/40001/rekall/plugins/addrspaces... rekall/plugins/addrspaces/amd64.py:364: p2m_top_location = self.session.profile.get_constant("p2m_top", False) p2m_top_location = self.session.profile.get_constant("p2m_top", "Pointer") p2m_address = p2m_top_location.deref().obj_offset or p2m_address = p2m_top_location.v() https://codereview.appspot.com/200130043/diff/40001/rekall/plugins/addrspaces... rekall/plugins/addrspaces/amd64.py:407: new_mapping[mfn] = pfn maybe here: for i, p2m_top in enumerate(self._ReadP2M(p2m_address, self.P2M_TOP_PER_PAGE)): self.session.report_progress("Building m2p map %s%%" % (100 * i / 64)) https://codereview.appspot.com/200130043/diff/40001/rekall/plugins/addrspaces... rekall/plugins/addrspaces/amd64.py:447: # mapping. Should we use this in every linux case? https://codereview.appspot.com/200130043/diff/40001/rekall/plugins/core.py File rekall/plugins/core.py (right): https://codereview.appspot.com/200130043/diff/40001/rekall/plugins/core.py#ne... rekall/plugins/core.py:281: "CONFIG_PARAVIRT_GUEST") == "y"): This just checks if the kernel is configured for xen and most distro kernels are current. You need to check the value of p2m_top to see if xen is actually enabled.
Sign in to reply to this message.
https://codereview.appspot.com/200130043/diff/40001/rekall/plugins/addrspaces... File rekall/plugins/addrspaces/amd64.py (right): https://codereview.appspot.com/200130043/diff/40001/rekall/plugins/addrspaces... rekall/plugins/addrspaces/amd64.py:316: On 2015/01/30 15:16:56, scudette wrote: > docstring Done. https://codereview.appspot.com/200130043/diff/40001/rekall/plugins/addrspaces... rekall/plugins/addrspaces/amd64.py:364: p2m_top_location = self.session.profile.get_constant("p2m_top", False) On 2015/01/30 15:16:56, scudette wrote: > p2m_top_location = self.session.profile.get_constant("p2m_top", "Pointer") > > p2m_address = p2m_top_location.deref().obj_offset > or > p2m_address = p2m_top_location.v() Done. https://codereview.appspot.com/200130043/diff/40001/rekall/plugins/addrspaces... rekall/plugins/addrspaces/amd64.py:407: new_mapping[mfn] = pfn On 2015/01/30 15:16:56, scudette wrote: > maybe here: > for i, p2m_top in enumerate(self._ReadP2M(p2m_address, self.P2M_TOP_PER_PAGE)): > self.session.report_progress("Building m2p map %s%%" % (100 * i / 64)) Done. https://codereview.appspot.com/200130043/diff/40001/rekall/plugins/addrspaces... rekall/plugins/addrspaces/amd64.py:447: # mapping. On 2015/01/30 15:16:56, scudette wrote: > Should we use this in every linux case? Maybe? There's the tradeoff here where we wouldn't see any manual alterations of the page tables because we bypass them. We may want to refactor this part in the future to make it an option. https://codereview.appspot.com/200130043/diff/40001/rekall/plugins/core.py File rekall/plugins/core.py (right): https://codereview.appspot.com/200130043/diff/40001/rekall/plugins/core.py#ne... rekall/plugins/core.py:281: "CONFIG_PARAVIRT_GUEST") == "y"): On 2015/01/30 15:16:56, scudette wrote: > This just checks if the kernel is configured for xen and most distro kernels are > current. You need to check the value of p2m_top to see if xen is actually > enabled. Done. I've had to move os-specific address space selection in find_dtb to the os-specific find_dtb plugins. NOTE: The configuration for XEN support is CONFIG_PARAVIRT. This config option is only active if we can be PV guest. But doesn't guarantee it is an active guest.
Sign in to reply to this message.
LGTM - Awesome work!
Sign in to reply to this message.
Thanks! Half the work on the CL is yours :)
Sign in to reply to this message.
Message was sent while issue was closed.
Committed patchset #5 manually as r366bc97 (presubmit successful).
Sign in to reply to this message.
Checking why this CL broke an entity test. On Sat, Jan 31, 2015 at 9:05 AM, <parki.san@gmail.com> wrote: > Committed patchset #5 manually as r366bc97 (presubmit successful). > > https://codereview.appspot.com/200130043/ >
Sign in to reply to this message.
+adam Maybe the TestFindEntity test is flaky? I reran the build and all passed. Locally I can't reproduce it. On Sat, Jan 31, 2015 at 9:13 AM, Jordi Sanchez <parki.san@gmail.com> wrote: > Checking why this CL broke an entity test. > > On Sat, Jan 31, 2015 at 9:05 AM, <parki.san@gmail.com> wrote: > >> Committed patchset #5 manually as r366bc97 (presubmit successful). >> >> https://codereview.appspot.com/200130043/ >> > >
Sign in to reply to this message.
you can try to re-run the test on travis. On Sat, Jan 31, 2015 at 9:27 AM, Jordi Sanchez <parki.san@gmail.com> wrote: > +adam > > Maybe the TestFindEntity test is flaky? I reran the build and all passed. > Locally I can't reproduce it. > > On Sat, Jan 31, 2015 at 9:13 AM, Jordi Sanchez <parki.san@gmail.com> wrote: >> >> Checking why this CL broke an entity test. >> >> On Sat, Jan 31, 2015 at 9:05 AM, <parki.san@gmail.com> wrote: >>> >>> Committed patchset #5 manually as r366bc97 (presubmit successful). >>> >>> https://codereview.appspot.com/200130043/ >> >> > > -- > You received this message because you are subscribed to the Google Groups > "rekall-dev" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to rekall-dev+unsubscribe@googlegroups.com. > To post to this group, send email to rekall-dev@googlegroups.com. > For more options, visit https://groups.google.com/d/optout.
Sign in to reply to this message.
Yeah, I did earlier. It's passing. On Sat, Jan 31, 2015, 11:23 AM 'Michael Cohen' via rekall-dev < rekall-dev@googlegroups.com> wrote: > you can try to re-run the test on travis. > > On Sat, Jan 31, 2015 at 9:27 AM, Jordi Sanchez <parki.san@gmail.com> > wrote: > > +adam > > > > Maybe the TestFindEntity test is flaky? I reran the build and all passed. > > Locally I can't reproduce it. > > > > On Sat, Jan 31, 2015 at 9:13 AM, Jordi Sanchez <parki.san@gmail.com> > wrote: > >> > >> Checking why this CL broke an entity test. > >> > >> On Sat, Jan 31, 2015 at 9:05 AM, <parki.san@gmail.com> wrote: > >>> > >>> Committed patchset #5 manually as r366bc97 (presubmit successful). > >>> > >>> https://codereview.appspot.com/200130043/ > >> > >> > > > > -- > > You received this message because you are subscribed to the Google Groups > > "rekall-dev" group. > > To unsubscribe from this group and stop receiving emails from it, send an > > email to rekall-dev+unsubscribe@googlegroups.com. > > To post to this group, send email to rekall-dev@googlegroups.com. > > For more options, visit https://groups.google.com/d/optout. > > -- > You received this message because you are subscribed to the Google Groups > "rekall-dev" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to rekall-dev+unsubscribe@googlegroups.com. > To post to this group, send email to rekall-dev@googlegroups.com. > For more options, visit https://groups.google.com/d/optout. >
Sign in to reply to this message.
sometimes travis tests fail because the connection to the profile repo failed for some reason. Our tests are not fully hermetic. On 31 January 2015 at 11:24, Jordi Sanchez <parki.san@gmail.com> wrote: > Yeah, I did earlier. It's passing. > > > On Sat, Jan 31, 2015, 11:23 AM 'Michael Cohen' via rekall-dev > <rekall-dev@googlegroups.com> wrote: >> >> you can try to re-run the test on travis. >> >> On Sat, Jan 31, 2015 at 9:27 AM, Jordi Sanchez <parki.san@gmail.com> >> wrote: >> > +adam >> > >> > Maybe the TestFindEntity test is flaky? I reran the build and all >> > passed. >> > Locally I can't reproduce it. >> > >> > On Sat, Jan 31, 2015 at 9:13 AM, Jordi Sanchez <parki.san@gmail.com> >> > wrote: >> >> >> >> Checking why this CL broke an entity test. >> >> >> >> On Sat, Jan 31, 2015 at 9:05 AM, <parki.san@gmail.com> wrote: >> >>> >> >>> Committed patchset #5 manually as r366bc97 (presubmit successful). >> >>> >> >>> https://codereview.appspot.com/200130043/ >> >> >> >> >> > >> > -- >> > You received this message because you are subscribed to the Google >> > Groups >> > "rekall-dev" group. >> > To unsubscribe from this group and stop receiving emails from it, send >> > an >> > email to rekall-dev+unsubscribe@googlegroups.com. >> > To post to this group, send email to rekall-dev@googlegroups.com. >> > For more options, visit https://groups.google.com/d/optout. >> >> -- >> You received this message because you are subscribed to the Google Groups >> "rekall-dev" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to rekall-dev+unsubscribe@googlegroups.com. >> To post to this group, send email to rekall-dev@googlegroups.com. >> For more options, visit https://groups.google.com/d/optout.
Sign in to reply to this message.
Message was sent while issue was closed.
On 2015/01/31 08:28:19, Jordi wrote: > +adam > > Maybe the TestFindEntity test is flaky? I reran the build and all passed. > Locally I can't reproduce it. > > On Sat, Jan 31, 2015 at 9:13 AM, Jordi Sanchez <mailto:parki.san@gmail.com> wrote: > > > Checking why this CL broke an entity test. > > > > On Sat, Jan 31, 2015 at 9:05 AM, <mailto:parki.san@gmail.com> wrote: > > > >> Committed patchset #5 manually as r366bc97 (presubmit successful). > >> > >> https://codereview.appspot.com/200130043/ > >> > > > > Do you have an example of how it broke Jordi? I don't see any TAP result for this issue.
Sign in to reply to this message.
Message was sent while issue was closed.
On 2015/01/31 13:13:31, adamsh wrote: > On 2015/01/31 08:28:19, Jordi wrote: > > +adam > > > > Maybe the TestFindEntity test is flaky? I reran the build and all passed. > > Locally I can't reproduce it. > > > > On Sat, Jan 31, 2015 at 9:13 AM, Jordi Sanchez <mailto:parki.san@gmail.com> > wrote: > > > > > Checking why this CL broke an entity test. > > > > > > On Sat, Jan 31, 2015 at 9:05 AM, <mailto:parki.san@gmail.com> wrote: > > > > > >> Committed patchset #5 manually as r366bc97 (presubmit successful). > > >> > > >> https://codereview.appspot.com/200130043/ > > >> > > > > > > > > Do you have an example of how it broke Jordi? I don't see any TAP result for > this issue. I also don't see any test called FindEntity.
Sign in to reply to this message.
Message was sent while issue was closed.
On 2015/01/31 13:16:09, the80srobot wrote: > On 2015/01/31 13:13:31, adamsh wrote: > > On 2015/01/31 08:28:19, Jordi wrote: > > > +adam > > > > > > Maybe the TestFindEntity test is flaky? I reran the build and all passed. > > > Locally I can't reproduce it. > > > > > > On Sat, Jan 31, 2015 at 9:13 AM, Jordi Sanchez <mailto:parki.san@gmail.com> > > wrote: > > > > > > > Checking why this CL broke an entity test. > > > > > > > > On Sat, Jan 31, 2015 at 9:05 AM, <mailto:parki.san@gmail.com> wrote: > > > > > > > >> Committed patchset #5 manually as r366bc97 (presubmit successful). > > > >> > > > >> https://codereview.appspot.com/200130043/ > > > >> > > > > > > > > > > > > Do you have an example of how it broke Jordi? I don't see any TAP result for > > this issue. > > I also don't see any test called FindEntity. Nevermind, found it - it's called EntityFind :) I don't see how that could be non-deterministic, given that the output is sorted. I'd be curious to see how it broke.
Sign in to reply to this message.
Looks like Travis doesn't keep old builds.... and we don't have access to the output... do we? >.< I remember an AssertionError: 2 != 1 But don't quite recall the backtrace soo... not very useful. On Sat, Jan 31, 2015 at 2:18 PM, <adam.sindelar@gmail.com> wrote: > On 2015/01/31 13:16:09, the80srobot wrote: > >> On 2015/01/31 13:13:31, adamsh wrote: >> > On 2015/01/31 08:28:19, Jordi wrote: >> > > +adam >> > > >> > > Maybe the TestFindEntity test is flaky? I reran the build and all >> > passed. > >> > > Locally I can't reproduce it. >> > > >> > > On Sat, Jan 31, 2015 at 9:13 AM, Jordi Sanchez >> > <mailto:parki.san@gmail.com> > >> > wrote: >> > > >> > > > Checking why this CL broke an entity test. >> > > > >> > > > On Sat, Jan 31, 2015 at 9:05 AM, <mailto:parki.san@gmail.com> >> > wrote: > >> > > > >> > > >> Committed patchset #5 manually as r366bc97 (presubmit >> > successful). > >> > > >> >> > > >> https://codereview.appspot.com/200130043/ >> > > >> >> > > > >> > > > >> > >> > Do you have an example of how it broke Jordi? I don't see any TAP >> > result for > >> > this issue. >> > > I also don't see any test called FindEntity. >> > > Nevermind, found it - it's called EntityFind :) I don't see how that > could be non-deterministic, given that the output is sorted. I'd be > curious to see how it broke. > > > https://codereview.appspot.com/200130043/ > > -- > You received this message because you are subscribed to the Google Groups > "rekall-dev" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to rekall-dev+unsubscribe@googlegroups.com. > To post to this group, send email to rekall-dev@googlegroups.com. > For more options, visit https://groups.google.com/d/optout. >
Sign in to reply to this message.
|
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
