#include <benchmarking.h>

Classes
struct	Options
	Options for profiling. More...

Public Types
enum	Metrics { WAYPOINTS = 1 << 0 , CORRECT = 1 << 1 , LENGTH = 1 << 2 , CLEARANCE = 1 << 3 , SMOOTHNESS = 1 << 4 }
	Bitmask options to select what metrics to compute for each run. More...

using	ComputeMetricCallback = std::function< PlannerMetric(const PlannerPtr &planner, const SceneConstPtr &scene, const planning_interface::MotionPlanRequest &request, const PlanData &run)>
	Type for callback function that returns a metric over the results of a planning query. More...

using	ProgressPropertyAllocator = std::function< Planner::ProgressProperty(const PlannerPtr &planner, const SceneConstPtr &scene, const planning_interface::MotionPlanRequest &request)>
	Allocator function that returns a planner progress property function for the current planning request. More...

using	ProgressCallback = std::function< void(const PlannerPtr &planner, const SceneConstPtr &scene, const planning_interface::MotionPlanRequest &request, const PlanData &result)>
	Type for callback function that is called in the planner progress property loop. More...

using	ProgressCallbackAllocator = std::function< ProgressCallback(const PlannerPtr &planner, const SceneConstPtr &scene, const planning_interface::MotionPlanRequest &request)>
	Allocator function that returns a progress property callback function for the current planning request. More...

Public Member Functions
bool	profilePlan (const PlannerPtr &planner, const SceneConstPtr &scene, const planning_interface::MotionPlanRequest &request, const Options &options, PlanData &result) const
	Profiling a single plan using a planner. More...

void	addMetricCallback (const std::string &name, const ComputeMetricCallback &metric)
	Add a callback function to compute a metric at the end of planning. More...

void	addProgressAllocator (const std::string &name, const ProgressPropertyAllocator &allocator)
	Add a function that allocates a function that returns a planner progress property function. More...

void	addProgressCallback (const ProgressCallback &callback)
	Add a function that is called in the planner progress property loop. More...

void	addProgressCallbackAllocator (const ProgressCallbackAllocator &allocator)
	Add a function that is called in the planner progress property loop. More...

Private Member Functions
void	computeBuiltinMetrics (uint32_t options, const SceneConstPtr &scene, PlanData &run) const
	Compute the built-in metrics according to the provided bitmask options. More...

void	computeCallbackMetrics (const PlannerPtr &planner, const SceneConstPtr &scene, const planning_interface::MotionPlanRequest &request, PlanData &run) const
	Compute the custom user callback metrics. More...

Private Attributes
std::map< std::string, ComputeMetricCallback >	callbacks_
	User callback metrics. More...

std::map< std::string, ProgressPropertyAllocator >	prog_allocators_
	User progress properties. More...

std::vector< ProgressCallback >	prog_callbacks_
	User progress callback functions. More...

std::vector< ProgressCallbackAllocator >	prog_callback_allocators_

Detailed Description

Definition at line 203 of file benchmarking.h.

Member Typedef Documentation

◆ ComputeMetricCallback

using robowflex::Profiler::ComputeMetricCallback = std::function<PlannerMetric(const PlannerPtr &planner, const SceneConstPtr &scene, const planning_interface::MotionPlanRequest &request, const PlanData &run)>

Type for callback function that returns a metric over the results of a planning query.

Parameters

[in]	planner	The planner being profiled.
[in]	scene	The scene used for planning.
[in]	request	The planning request.
[in]	run	The results of the planning run, including prior computed metrics.

Returns: The value of this metric. Will be stored in run data under associated name.

Definition at line 234 of file benchmarking.h.

◆ ProgressCallback

using robowflex::Profiler::ProgressCallback = std::function<void(const PlannerPtr &planner, const SceneConstPtr &scene, const planning_interface::MotionPlanRequest &request, const PlanData &result)>

Type for callback function that is called in the planner progress property loop.

Parameters

[in]	planner	The planner being profiled.
[in]	scene	The scene used for planning.
[in]	request	The planning request.
[in]	run	The results of the planning run, including prior computed metrics.

Definition at line 258 of file benchmarking.h.

◆ ProgressCallbackAllocator

using robowflex::Profiler::ProgressCallbackAllocator = std::function<ProgressCallback(const PlannerPtr &planner, const SceneConstPtr &scene, const planning_interface::MotionPlanRequest &request)>

Allocator function that returns a progress property callback function for the current planning request.

Parameters

[in]	planner	The planner being profiled.
[in]	scene	The scene used for planning.
[in]	request	The planning request.

Returns: A progress callback function.

Definition at line 270 of file benchmarking.h.

◆ ProgressPropertyAllocator

using robowflex::Profiler::ProgressPropertyAllocator = std::function<Planner::ProgressProperty(const PlannerPtr &planner, const SceneConstPtr &scene, const planning_interface::MotionPlanRequest &request)>

Allocator function that returns a planner progress property function for the current planning request.

Parameters

[in]	planner	The planner being profiled.
[in]	scene	The scene used for planning.
[in]	request	The planning request.

Returns: A planner progress property function.

Definition at line 247 of file benchmarking.h.

Member Enumeration Documentation

◆ Metrics

enum robowflex::Profiler::Metrics

Bitmask options to select what metrics to compute for each run.

Enumerator
WAYPOINTS	Number of waypoints in path.
CORRECT	Is the path correct (no collisions?).
LENGTH	Length of the path.
CLEARANCE	Clearance of path from obstacles.
SMOOTHNESS	Smoothness of path.

Definition at line 208 of file benchmarking.h.

         {
             WAYPOINTS = 1 << 0,   ///< Number of waypoints in path.
             CORRECT = 1 << 1,     ///< Is the path correct (no collisions?).
             LENGTH = 1 << 2,      ///< Length of the path.
             CLEARANCE = 1 << 3,   ///< Clearance of path from obstacles.
             SMOOTHNESS = 1 << 4,  ///< Smoothness of path.
         };

Member Function Documentation

◆ addMetricCallback()

void Profiler::addMetricCallback	(	const std::string &	name,
		const ComputeMetricCallback &	metric
	)

Add a callback function to compute a metric at the end of planning.

Parameters

[in]	name	Name of the metric.
[in]	metric	Function to use for callback.

Definition at line 245 of file benchmarking.cpp.

 {
     callbacks_.emplace(name, metric);
 }

◆ addProgressAllocator()

void Profiler::addProgressAllocator	(	const std::string &	name,
		const ProgressPropertyAllocator &	allocator
	)

Add a function that allocates a function that returns a planner progress property function.

Parameters

[in]	name	Name of the planner progress property.
[in]	allocator	Allocator function.

Definition at line 250 of file benchmarking.cpp.

 {
     prog_allocators_.emplace(name, allocator);
 }

◆ addProgressCallback()

void Profiler::addProgressCallback ( const ProgressCallback & callback )

Add a function that is called in the planner progress property loop.

Parameters

[in] callback Callback function to add.

Definition at line 255 of file benchmarking.cpp.

 {
     prog_callbacks_.emplace_back(callback);
 }

◆ addProgressCallbackAllocator()

void Profiler::addProgressCallbackAllocator ( const ProgressCallbackAllocator & allocator )

Add a function that is called in the planner progress property loop.

Parameters

[in] callback Callback function to add.

Definition at line 260 of file benchmarking.cpp.

 {
     prog_callback_allocators_.emplace_back(allocator);
 }

◆ computeBuiltinMetrics()

void Profiler::computeBuiltinMetrics	(	uint32_t	options,
		const SceneConstPtr &	scene,
		PlanData &	run
	)		const

private

Compute the built-in metrics according to the provided bitmask options.

Parameters

[in]	options	Bitmask of which built-in metrics to compute.
[in]	scene	Scene used for planning and metric computation.
[out]	run	Metric results.

Definition at line 265 of file benchmarking.cpp.

 {
     if (options & Metrics::WAYPOINTS)
         run.metrics["waypoints"] = run.success ? int(run.trajectory->getNumWaypoints()) : int(0);
  
     if (options & Metrics::LENGTH)
         run.metrics["length"] = run.success ? run.trajectory->getLength() : 0.0;
  
     if (options & Metrics::CORRECT)
         run.metrics["correct"] = run.success ? run.trajectory->isCollisionFree(scene) : false;
  
     if (options & Metrics::CLEARANCE)
         run.metrics["clearance"] = run.success ? std::get<0>(run.trajectory->getClearance(scene)) : 0.0;
  
     if (options & Metrics::SMOOTHNESS)
         run.metrics["smoothness"] = run.success ? run.trajectory->getSmoothness() : 0.0;
  
     run.metrics["robowflex_planner_name"] = run.query.planner->getName();
     run.metrics["robowflex_robot_name"] = run.query.planner->getRobot()->getName();
  
     run.metrics["request_planner_type"] = std::string(ROBOWFLEX_DEMANGLE(typeid(*run.query.planner).name()));
     run.metrics["request_planner_id"] = run.query.request.planner_id;
     run.metrics["request_group_name"] = run.query.request.group_name;
     run.metrics["request_num_planning_attempts"] = run.query.request.num_planning_attempts;
  
     run.metrics["machine_hostname"] = run.hostname;
     run.metrics["machine_thread_id"] = run.thread_id;
     run.metrics["machine_process_id"] = run.process_id;
 }

◆ computeCallbackMetrics()

void Profiler::computeCallbackMetrics	(	const PlannerPtr &	planner,
		const SceneConstPtr &	scene,
		const planning_interface::MotionPlanRequest &	request,
		PlanData &	run
	)		const

private

Compute the custom user callback metrics.

Parameters

[in]	planner	Planner used.
[in]	scene	Scene used for planning.
[in]	request	Planning request.
[out]	run	Metric results.

Definition at line 295 of file benchmarking.cpp.

 {
     for (const auto &callback : callbacks_)
         run.metrics[callback.first] = callback.second(planner, scene, request, run);
 }

◆ profilePlan()

bool Profiler::profilePlan	(	const PlannerPtr &	planner,
		const SceneConstPtr &	scene,
		const planning_interface::MotionPlanRequest &	request,
		const Options &	options,
		PlanData &	result
	)		const

Profiling a single plan using a planner.

Parameters

[in]	planner	Planner to profile.
[in]	scene	Scene to plan in.
[in]	request	Planning request to profile.
[in]	options	The options for profiling.
[out]	result	The results of profiling.

Returns: True if planning succeeded, false on failure.

Profiler

Definition at line 137 of file benchmarking.cpp.

 {
     bool complete = false;
     std::mutex mutex;
     std::shared_ptr<std::thread> progress_thread;
  
     result.query.scene = scene;
     result.query.planner = planner;
     result.query.request = request;
  
     result.start = IO::getDate();
  
     // Setup planner progress property thread
     std::map<std::string, Planner::ProgressProperty> prog_props;
     const auto &pp = planner->getProgressProperties(scene, request);
     prog_props.insert(pp.begin(), pp.end());
  
     // Add custom progress properties
     for (const auto &allocator : prog_allocators_)
         prog_props.emplace(allocator.first, allocator.second(planner, scene, request));
  
     // Setup progress callbacks
     std::vector<ProgressCallback> prog_call;
     prog_call.insert(prog_call.end(), prog_callbacks_.begin(), prog_callbacks_.end());
  
     for (const auto &allocator : prog_callback_allocators_)
         prog_call.emplace_back(allocator(planner, scene, request));
  
     const bool have_prog = not prog_props.empty();
     if (options.progress  //
         and (have_prog or not prog_call.empty()))
     {
         // Get names of progress properties
         if (have_prog)
         {
             for (const auto &property : prog_props)
                 result.property_names.emplace_back(property.first);
             result.property_names.emplace_back("time REAL");
         }
  
         progress_thread.reset(new std::thread([&] {
             bool at_least_once = options.progress_at_least_once;
             while (true)
             {
                 // Sleep until the next update
                 IO::threadSleep(options.progress_update_rate);
  
                 std::unique_lock<std::mutex> lock(mutex);
                 if (not at_least_once and complete)
                     return;
  
                 if (have_prog)
                 {
                     std::map<std::string, std::string> data;
  
                     // Add time stamp
                     double time = IO::getSeconds(result.start, IO::getDate());
                     data["time REAL"] = std::to_string(time);
  
                     // Compute properties
                     for (const auto &property : prog_props)
                         data[property.first] = property.second();
  
                     result.progress.emplace_back(data);
                 }
  
                 for (const auto &callback : prog_call)
                     callback(planner, scene, request, result);
  
                 at_least_once = false;
             }
         }));
     }
  
     // Plan
     result.response = planner->plan(scene, request);
  
     // Notify planner progress thread
     {
         std::unique_lock<std::mutex> lock(mutex);
         complete = true;
     }
  
     // Compute metrics and fill out results
     result.finish = IO::getDate();
     result.time = IO::getSeconds(result.start, result.finish);
     result.success = result.response.error_code_.val == moveit_msgs::MoveItErrorCodes::SUCCESS;
  
     if (result.success)
         result.trajectory = std::make_shared<Trajectory>(*result.response.trajectory_);
  
     result.hostname = IO::getHostname();
     result.process_id = IO::getProcessID();
     result.thread_id = IO::getThreadID();
  
     computeBuiltinMetrics(options.metrics, scene, result);
     computeCallbackMetrics(planner, scene, request, result);
  
     if (progress_thread)
         progress_thread->join();
  
     return result.success;
 }

Member Data Documentation

◆ callbacks_

std::map<std::string, ComputeMetricCallback> robowflex::Profiler::callbacks_

private

User callback metrics.

Definition at line 330 of file benchmarking.h.

◆ prog_allocators_

std::map<std::string, ProgressPropertyAllocator> robowflex::Profiler::prog_allocators_

private

User progress properties.

Definition at line 331 of file benchmarking.h.

◆ prog_callback_allocators_

std::vector<ProgressCallbackAllocator> robowflex::Profiler::prog_callback_allocators_

private

User progress callback function allocators.

Definition at line 333 of file benchmarking.h.

◆ prog_callbacks_

std::vector<ProgressCallback> robowflex::Profiler::prog_callbacks_

private

User progress callback functions.

Definition at line 332 of file benchmarking.h.

The documentation for this class was generated from the following files:

Classes

Public Types

Public Member Functions

Private Member Functions

Private Attributes

Detailed Description

Member Typedef Documentation

◆ ComputeMetricCallback

◆ ProgressCallback

◆ ProgressCallbackAllocator

◆ ProgressPropertyAllocator

Member Enumeration Documentation

◆ Metrics

Member Function Documentation

◆ addMetricCallback()

◆ addProgressAllocator()

◆ addProgressCallback()

◆ addProgressCallbackAllocator()

◆ computeBuiltinMetrics()

◆ computeCallbackMetrics()

◆ profilePlan()

Member Data Documentation

◆ callbacks_

◆ prog_allocators_

◆ prog_callback_allocators_

◆ prog_callbacks_