Chimbuko: Performance Analysis

• Performance analysis C/C++ library

• This library is a part of Chimbuko, a workflow-level scalable performance trace analysis tool.

• Funded by the Exascale Computing Project (ECP), U.S. Department of Energy

Related Github repositories

• Chimbuko

• Performance Analysis

• Visualization

Introduction

• Overview
• On-node AD Module
  • Parser
  • Pre-processing
  • Anomaly detection
    • Anomaly detection algorithm
    • Advanced anomaly analysis
  • Provenance data collection
  • Stream local vizualisation data
  • Post-processing
• Parameter Server
  • Design
  • Anomaly ranking metrics
• Provenance Database

Installation and Usage

• Installation and Usage
• Installation
  • Basic installation
    • A note on libfabric providers
    • Integrating with system-installed MPI
    • Non-MPI installation (advanced)
  • Summit
  • Spock
• Instrumenting an application with Tau
  • C++
  • CUDA/C++
  • Python
• Running an application under Chimbuko
  • Online Analysis
    • Running on Summit
      • ERF files
      • URS files
    • Running on Spock
    • Scaling to large job sizes
    • Online analysis of an MPI application with a non-MPI installation of Chimbuko (advanced)
    • Online analysis of a non-MPI application with a non-MPI installation of Chimbuko (advanced)
  • Examples
    • NWChem
    • MOCU (ExaLearn)
  • Interacting with the Provenance Database

IO Schema

• Provenance Database Schema
  • Main database
    • Function event schema
    • Metadata schema
  • Global database
    • Function profile statistics schema
    • Counter statistics schema
    • AD model schema
• Parameter Server Streaming Output

Appendix

• Installation
  • Gtest Installation
  • ADIOS2 Build Location
  • GoogleTest on Summit
  • Manual installation of Chimbuko
• Instrumentation with TAU
  • Environment Variables TAU
• Usage
  • Chimbuko Config
  • Launch Services
  • Additional ProvDB Variables
  • Visualization Variables
  • Parameter Server Variables
  • Additional pserver Variables
  • AD Variables
  • Additional AD Variables
  • Offline Analysis

API For Developers

• Full API Listing
  • API
    • AD
      • chimbuko
      • ADAnomalyProvenance
      • ADCounter
      • ADDefine
      • ADEvent
      • ADglobalFunctionIndexMap
      • ADio
      • ADLocalCounterStatistics
      • ADLocalFuncStatistics
      • ADMetadataParser
      • ADNetClient
      • ADNormalEventProvenance
      • ADOutlier
      • ADParser
      • ADProvenanceDBclient
      • ADProvenanceDBengine
      • AnomalyData
      • ExecData
      • utils
    • Anomaly Detection Algorithm Parameters
      • ParamInterface
      • CopodParam
      • HbosParam
      • SstdParam
    • Parameter Server
      • AnomalyStat
      • global_anomaly_stats
      • global_counter_stats
      • PSglobalFunctionIndexMap
      • PSProvenanceDBclient
      • PSstatSender
    • Network
      • NetInterface
      • MPINet
      • ZMQNet
      • ZMQMENet
    • Message
    • Utils
      • ADIOS2parseUtils
      • Anomalies
      • barrier
      • commandLineParser
      • DispatchQueue
      • error
      • hash
      • map
      • memutils
      • mtQueue
      • PerfStats
      • RunMetric
      • RunStats
      • string
      • threadPool
      • time
      • verbose

Indices and tables

• Index

• Module Index

• Search Page