ROSE Compiler Framework/OpenK

Overview

An ongoing project to explore knowledge-driven HPC analysis and optimization. We use the standard and toolchain used by OWL to formally model the concepts and relations in HPC domains, including programs, hardware, analysis and optimization, etc.

The framework has several parts:

One is the C program parser to generate database.
- The source code is under OpenK/tools/rosePrgKnowledgeBuilder, it extracts the knowledge from the input program and store it as ontology in turtle format.
- The output knowledge base can be parsed with SWI-Prolog semweb library.
The second part is the canonical loop analysis part written in Prolog.
The third part is a demo of using Prolog C++ interface to query hardware ontology

OpenK has been integrated into ROSE under

https://github.com/rose-compiler/rose/tree/master/projects/OpenK

The documentation below is a bit out of date and needs to be updated.

Directory Layout

List

ontology/ - owl files.
projects/ - contains independent projects or programs: canonical loop detection, pointer analysis, etc.
- canonicalloop/ - using Prolog query to find canonical loops
- cfg_test/ - using Prolog to generate control flow graph
- rose_canonicalloop/ - classic AST-based canonical loop detection, used for comparison
- staticCFG_tests/- classic AST-based control flow graph generation, used for comparison
test/ - test benchmarks and scripts
tools/ - the knowledge generator: use ROSE frontend to parse the C code and build a knowledge base.

Software Dependences

You need to install

ROSE
SWI-Prolog

User Guide

To build the translator:

Set ROSE environment by $ . openk/set.rose . Note the set.rose file should be modified according to your machine.
1. Or just manually export ROSE_INS=/home/liao6/workspace/masterDevClean/install
2. export BOOST_INS=/nfs/casc/overture/ROSE/opt/rhel7/x86_64/boost/1_54_0/gcc/4.8.3
cd to openk/tools/rosePrgKnowledgeBuilder folder.
You can run make to make the rosePrgKnowledgeBuilder.exe and make check to check it against test file under the same directory.

rosePrgKnowledgeBuilder.exe

./rosePrgKnowledgeBuilder.exe -emit-ttl ut.cc.ttl -c ./ut.cc

openk.git/tools/rosePrgKnowledgeBuilder]./rosePrgKnowledgeBuilder.exe --help

---------------------Tool-Specific Help-----------------------------------
This is a source translator to build knowledge base from your C/C++ code.
Usage: ./rosePrgKnowledgeBuilder.exe[options][-o output]-c input.c

Options:
        --help              This help message.
        -emit-pl output.pl       ontologies in prolog file.
        -emit-owl output.owl     ontologies in OWL format.
        -emit-ttl output.ttl     ontologies in Turtle.
        -alive true/false        embedding Prolog engine.

Declarative canonical loop analysis

A declarative analysis tool is in openk/projects/canonicalloop

The openk/csem is the prolog modules that implement C program fundamental analysis including searching different kinds of program constructs. They can be seen as the library modules. Feel free to add more modules under csem to extend the library.
Every prolog-based c program analysis should have its own directory with a run.pl as the driver program. They use modules in csem.

For example, the canonical loop analysis is under openk/projects/canonicalloop.

Run swipl -s run.pl example_knowledge_base.ttl output_report.txt

A reference rose native C++ implementation of the canonical loop is under openk/projects/rose_canonicalloop.

Run make to make roseNativeCl.exe

To test it against testbench:

cd openk/test/npb2.3-test/
There is a script for invoking the test. The script uses the tools rosePrgKnowledgeBuilder.exe and roseNativeCl.exe and the prolog program. You don't need to do anything, the script uses the relative path to these tools.
$chmod +x analysis.py
./analysis.py rose-cl to run the native Rose canonical loop execution. The output logs will be generated under current folder.
./analysis.py build-kb to run the parser to generate the database, then ./analysis openk-cl to run the prolog-based canonical loop on all test files.
Or simple, you can just run ./analysis.py all to run all the tests.

declarative control flow graph

cd projects/cfg_test

To generate a CFG report from a ttl file

swipl --nosignal --quiet run.pl sp_single.c.ttl sp_single.c.ttl_openk_cfg_report.txt

hardware ontology demo

openk.git/projects/owl_hw_demo

hpchardware-rdf.owl HPC hardware ontology
load2.pl prolog predicates to read ontology via prolog semantic web library
main.cpp C++ program interacting with the ontology via SWI-Prolog C++ interface

testing

We provide a python script to run various tests: cd /test/npb2.3-test

./run_test.py <option>

Option can be one of the following:

clean: clean the result files.
build-kb: build knowledge base from the input codes
openk-cl: run the openk canonical loop analysis. Must after the build-kb step
rose-cl: run the rose native canonical loop analysis
openk-cfg: openk cfg

key source files

List

openk.git/tools/rosePrgKnowledgeBuilder/
- rosePrgKnowledgeBuilder.cpp
- OntoGeneratorInterface.hpp

Command line:

swipl --nosignal --quiet /export/liao6/openk.git/projects/canonicalloop/run.pl bt.c.ttl bt.c.pl_openk_cl_report.txt

./projects/canonicalloop/run.pl

:- use_module(library(semweb/rdf_db)).
:- use_module(library(semweb/turtle)).
:- use_module('../csem/canonical_loop.pl').
:- use_module('../csem/url_process.pl').

./projects/csem/canonical_loop.pl

/* CanonicalLoop */
:- module(canonical_loop, [canonicalLoop/1]).
:- use_module(c_construct).

%%	The top level specification
canonicalLoop(LoopURL) :-
	forLoop(LoopURL),
	hasForInit(LoopURL, Init),
	isCanonicalInit(Init, LoopVar),

	hasForTest(LoopURL, Test),
	isCanonicalTest(Test, LoopVar),

	hasForIncr(LoopURL, Incr),
	isCanonicalIncr(Incr, LoopVar).

isCanonicalInit(InitURL, LoopVar) :-
	%% forLoop(LoopURL), hasForInit(LoopURL, InitURL),
	roseCanonicalInit(InitURL, LoopVar, _),
	hasType(LoopVar, Type), 
	(intType(Type); Type == pointer_type).

%% normal style
roseCanonicalInit(InitURL, LoopVar, LB) :-
	hasChild(InitURL, AssignOpURL),
	c_is_a(AssignOpURL, 'AssignOp'), !,
	leftOperand(AssignOpURL, VarRefURL),
	get_varDecl(VarRefURL, LoopVar),
	rightOperand(AssignOpURL, LB).

%% C99 style
%% In this case, rose AST nodes have the same location
roseCanonicalInit(InitURL, LoopVar, LB) :-
	%% hasChild(InitURL, VarURL),
	c_is_a(InitURL, 'VariableDecl'),
	c_is_a(InitURL, 'Variable'), !,
	\+ (hasChild(InitURL, VarDecl), c_is_a(VarDecl, 'VariableDecl')),
	roseHasInitValue(InitURL, LB),
	LoopVar = InitURL.
...