ROSE Compiler Framework/OpenMP Support

Documentation

Official documentations

• The ROSE manual has a chapter (Chapter 12 OpenMP Support) explaining the details. pdf
• A paper was published for the uniqueness of the ROSE OpenMP Implementation pdf
• A paper about the Accelerator (GPU) support: "Early experiences with the openmp accelerator model."

OpenMP Features

ROSE supports OpenMP 3.0 for C/C++ (and limited Fortran support) and has an experimental OpenMP Acclerator Model Implementation from OpenMP 4.0

• OpenMP Acclerator Model Implementation

Connection point to the frontend() is located in sage_support.cpp, via OmpSupport::processOpenMP() called by SgFile::secondaryPassOverSourceFile()

void processOpenMP(SgSourceFile *sageFilePtr) of ompAstConstruction.cpp // top level driver of OpenMP processing in ROSE

• Frontend parsing source files (ompparser.yy and ompFortranParser.C) are located under https://github.com/rose-compiler/rose/tree/master/src/frontend/SageIII
  • void build_OpenMP_AST(SgSourceFile *sageFilePtr) of ompAstConstruction.cpp // process pragmas and build AST for OpenMP
    • void attachOmpAttributeInfo(SgSourceFile *sageFilePtr) of ompAstConstruction.cpp // find and parse OpenMP pragmas into OmpAttribute info.
    • convert_OpenMP_pragma_to_AST() // convert attributes into dedicated OpenMP AST nodes
• The transformation of OpenMP into threaded code is located in omp_lowering.cpp, under https://github.com/rose-compiler/rose/blob/master/src/midend/programTransformation/ompLowering
  • lower_omp()
• The OpenMP runtime interface is defined in libxomp.h and xomp.c under the same ompLowering directory mentioned above
  • GCC's OpenMP runtime library, libgomp is used with this implementation.
  • Optionally, other runtime libraries, such as Omni runtime library can be hooked with ROSE. But this support is not being actively maintained.

The OpenMP processing happens in void SgFile::secondaryPassOverSourceFile() of rose/src/frontend/SageIII/sage_support/sage_support.cpp. This pass collects extra information about the source file such as comments, preprocessing directives, etc.

Configuration

Configuration: please always try to use --with-gomp_omp_runtime_library=/usr/apps/gcc/4.4.1/lib64/ when configuring ROSE. So the generated ROSE translators can automatically link with libgomp.a to generate executables for you. This will also allow the execution tests of the omp Lowering be executed to catch errors. Without this option, only the compile level tests will run.

Command line options

Like all other compilers supporting OpenMP, you have to explicitly turn on this support by using -rose:openmp:lowering, for example:

 identityTranslator -rose:openmp:lowering your_openmp_code.c

This option tells a ROSE translator to recognize OpenMP pragmas and translate the input to use libgomp runtime library. Detailed command lines for compilation, backend compiler , and linking can be seen by using a verbose flag

 identityTranslator -rose:openmp:lowering  -rose:verbose 3 your_openmp_code.c

Three internal steps will show up in this verbose mode:

• ROSE translator's translation: identityTranslator -rose:openmp:lowering -rose:verbose 3 ...-DUSE_ROSE --c -D_OPENMP your_openmp_code.c
• Backend compiler compilation: gcc -DUSE_ROSE -D_OPENMP -I/export/tmp.liao6/workspace/masterClean/build64/install/include rose_your_openmp_code.c -c -o mg_your_openmp_code.o
• Linking: gcc your_openmp_code.o -lm /export/tmp.liao6/workspace/masterClean/build64/install/lib/libxomp.a /usr/apps/gcc/4.4.1/lib64//libgomp.a -lpthread

More command line option choices for OpenMP support

• -rose:OpenMP, -rose:openmp follow OpenMP 3.0 specification for C/C++ and Fortran, perform one of the following actions:
• -rose:OpenMP:parse_only, -rose:openmp:parse_only parse OpenMP directives to OmpAttributes, no further actions (default behavior now)
• -rose:OpenMP:ast_only, -rose:openmp:ast_only on top of -rose:openmp:parse_only, build OpenMP AST nodes from OmpAttributes, no further actions
• -rose:OpenMP:lowering, -rose:openmp:lowering on top of -rose:openmp:ast_only, transform AST with OpenMP nodes into multithreaded code targeting GCC GOMP runtime library

internal steps

compilation

• ./identityTranslator -I../../../../../../rose-develop/src/frontend/SageIII -I../../../../../../rose-develop/src/midend/programTransformation/ompLowering -I../../../../.. -rose:openmp:lowering -g --edg:no_warnings -c ../../../../../../rose-develop/tests/nonsmoke/functional/CompileTests/OpenMP_tests/pi.c

Linking:

gcc pi.o -o pi.out -L../../../../../src/midend -lxomp /usr/lib/gcc/x86_64-linux-gnu/4.9//libgomp.a -lpthread -lm

Testing

Testing

• There are about 70 builtin execution tests (many have self-verification) in ROSE.
  • OpenMP parsing tests: tests/CompileTests/OpenMP_tests
  • OpenMP translation and execution tests: rose/tests/roseTests/ompLoweringTests, need --with-gomp_omp_runtime_library=/usr/apps/gcc/4.4.x(or later)/lib64/ to link and run the tests.

Some benchmarks are used to test OpenMP support in ROSE in Jenkins (our regression test server)

• a22b-NPB-2.3-C-parallel: all 8 benchmarks pass
• a21-SPEC-OMP-64bit-parallel: 3 benchmarks pass.
• LULESH OpenMP version: download

For builtin test:

• source files are in: https://github.com/rose-compiler/rose/tree/master/tests/nonsmoke/functional/CompileTests/OpenMP_tests
• the test driver is defined in https://github.com/rose-compiler/rose/tree/master/tests/nonsmoke/functional/roseTests/ompLoweringTests/Makefile.am . See the list of files under C_TESTCODES_REQUIRED_TO_RUN .

You have to configure the path to GOMP if you want to them to be automatically executed when "make check" is typed. e.g. ../sourcetree ... --with-gomp_omp_runtime_library=/usr/apps/gcc/4.4.1/lib64/

Parsing OpenMP directives

Frontend parsing source files (ompparser.yy and ompFortranParser.C) are located under https://github.com/rose-compiler/rose/tree/master/src/frontend/SageIII

• C/C++ directives are parsed using Flex/Bison:
  • Lexer https://github.com/rose-compiler/rose/blob/master/src/frontend/SageIII/omplexer.ll
  • https://github.com/rose-compiler/rose/blob/master/src/frontend/SageIII/ompparser.yy
• Fortran directives are parsed by a hand-written recursive descent parser https://github.com/rose-compiler/rose/blob/master/src/frontend/SageIII/ompFortranParser.C
• The driver to call the parsers: void processingOpenMP(SgSoureFile*) from https://github.com/rose-compiler/rose/blob/master/src/frontend/SageIII/ompAstConstruction.cpp
  • void attachOmpAttributeInfo(SgSourceFile *sageFilePtr) // call parse_fortran_openmp()

To recognize a new keyword, for example, a clause : target ()

• add a token name in ompparser.yy : %token TARGET
• parse the token in omplexer.ll: target {return cond_return ( TARGET ); }
• add grammar rules into ompparser.yy

openmp_directive : parallel_directive 
...
          | target_directive
....

target_directive: /* #pragma */ OMP TARGET {
                       ompattribute = buildOmpAttribute(e_target,gNode,true);
                       omptype = e_target;
                     }
                     target_clause_optseq 
                   ;

OmpAttribute

The parsing results are stored into persistent AST attribute. A derived type called OmpAttribute is used.

./src/frontend/SageIII/OmpAttribute.h /.C

AST support for OpenMP

We define a set of dedicated AST nodes to represent OpenMP directives in your source code. For an example input code, you can view the AST using dotGeneratorWholeASTGraph

/* A kernel for two level parallelizable loop with reduction */
float u[100][100];
float foo ()
{
  int i,j;
  float temp, error;
#pragma omp parallel for private (temp,i,j) reduction (+:error)
  for (i = 0; i < 100; i++)
    for (j = 0; j < 100; j++)
      {
        temp = u[i][j];
        error = error + temp * temp;
      }
  return error;
}

./dotGeneratorWholeASTGraph -c -rose:OpenMP:ast_only reduction_1.c

The generated dot graph will have the following nodes

• SgBasicBlock // the basic block of the function definition
  • SgOmpParallelStatement // dedicated OpenMP parallel statement in AST
    • SgOmpForStatement // dedicated Omp for statement. It has SgOmpReductionClause and SgOmpPrivateClause, obtained from ->get_clauses()
      • SgForStatement // the actual loop affected by "omp parallel for"

The combined directives like "omp parallel for " is split into two directives "omp parallel" and "omp for". All clauses are attached to the inner level directive as much as possible. This is to support simpler handling later on (like in OpenMP lowering).

• Note: there is no dedicated AST nodes for combined directives for now. But we can add them if necessary.

Omp Nodes

A list of the OpenMP AST nodes are (from http://rosecompiler.org/ROSE_HTML_Reference/annotated.html)

• SgOmpAlignedClause
• SgOmpAtomicStatement
• SgOmpBarrierStatement
• SgOmpBeginClause
• SgOmpBodyStatement
• SgOmpClause
• SgOmpClauseBodyStatement
• SgOmpCollapseClause
• SgOmpCopyinClause
• SgOmpCopyprivateClause
• SgOmpCriticalStatement
• SgOmpDefaultClause
• SgOmpDeviceClause
• SgOmpDoStatement
• SgOmpEndClause
• SgOmpExpressionClause
• SgOmpFirstprivateClause
• SgOmpFlushStatement
• SgOmpForStatement
• SgOmpIfClause
• SgOmpLastprivateClause
• SgOmpLinearClause
• SgOmpMapClause
• SgOmpMasterStatement
• SgOmpNowaitClause
• SgOmpNumThreadsClause
• SgOmpOrderedClause
• SgOmpOrderedStatement
• SgOmpParallelStatement
• SgOmpPrivateClause
• SgOmpReductionClause
• SgOmpSafelenClause
• SgOmpScheduleClause
• SgOmpSectionsStatement
• SgOmpSectionStatement
• SgOmpSharedClause
• SgOmpSimdStatement
• SgOmpSingleStatement
• SgOmpTargetDataStatement
• SgOmpTargetStatement
• SgOmpTaskStatement
• SgOmpTaskwaitStatement
• SgOmpThreadprivateStatement
• SgOmpUniformClause
• SgOmpUntiedClause
• SgOmpVariablesClause
• SgOmpWorkshareStatement

Please refer to the Doxygen documentation for the member access functions.

AST Graph

To view the AST graphs of your input OpenMP codes, you should use either dotGraphGeneratorWhole of pdfGenerator .

One example dot graph is available at parallefor.dot.png within https://github.com/chunhualiao/rose-ast .

Access OpenMP clauses and variables

Please refer to the Doxygen documentation for the member access functions of SgOmp* nodes.

Many helper functions are defined in the OmpSupport namespace, as shown in http://rosecompiler.org/ROSE_HTML_Reference/namespaceOmpSupport.html

• https://github.com/rose-compiler/rose/blob/master/src/frontend/SageIII/OmpAttribute.h
• https://github.com/rose-compiler/rose/blob/master/src/midend/programTransformation/ompLowering/omp_lowering.h

The best examples of how to access those nodes' information are in the OpenMP lowering step of ROSE.

• https://github.com/rose-compiler/rose/blob/master/src/midend/programTransformation/ompLowering/omp_lowering.cpp

For example, to obtain the list of variable references in threadprivate clause, the code looks like

  void transOmpThreadprivate(SgNode * node)
  {
    ROSE_ASSERT(node != NULL );
    SgOmpThreadprivateStatement* target = isSgOmpThreadprivateStatement(node);
    ROSE_ASSERT(target != NULL );

    SgVarRefExpPtrList nameList = target->get_variables ();
    for (size_t i = 0; i<nameList.size(); i++)
    {
      SgInitializedName* init_name = nameList[i]->get_symbol()->get_declaration();
      ROSE_ASSERT(init_name != NULL);
      SgVariableDeclaration*  decl = isSgVariableDeclaration(init_name-> get_declaration());
      ROSE_ASSERT (decl != NULL);
     ....
    }
   .... 
  }

Another example to check if a variable is within some OpenMP clauses' variable list

#include "rose.h"
#include "omp_lowering.h"  // many helper functions are declared here

using namespace OmpSupport; 

...
 SgInitializedName* orig_var = ... ;
 SgOmpClauseBodyStatement* clause_stmt = isSgOmpClauseBodyStatement(ompStmt);

// check if orig_var is within the specified clauses of an OpenMP statement (e.g. omp for)
  VariantVector vvt (V_SgOmpPrivateClause);
  vvt.push_back(V_SgOmpReductionClause);
  vvt.push_back(V_SgOmpFirstprivateClause);
  vvt.push_back(V_SgOmpLastprivateClause);

if (isInClauseVariableList(orig_var, clause_stmt, vvt))
{
...
}

// another one: check if a loop index variable is already in private() or not

isPrivateInRegion = isInClauseVariableList(index_var, isSgOmpClauseBodyStatement(omp_stmt), V_SgOmpPrivateClause);


// add loop index variable into the private() clause
  addClauseVariable(index_var,isSgOmpClauseBodyStatement(omp_loop), V_SgOmpPrivateClause);          

Query data sharing attributes

There is an interface function to query data sharing attribute of a variable referenced in an OpenMP program, declared in https://github.com/rose-compiler/rose/blob/master/src/midend/programTransformation/ompLowering/omp_lowering.h

namespace OmpSupport {

  //! Return the data sharing attribute type of a variable, specified as a symbol and an anchor node 
  //! (Must be the inner most node associated with the variable reference, e.g. a SgVarRefExp, SgVariableDeclaration, etc)
  //! Possible returned values include: e_shared, e_private,  e_firstprivate,  e_lastprivate,  e_reduction, 
  //! e_threadprivate, e_copyin, and e_copyprivate.
  ROSE_DLL_API omp_construct_enum getDataSharingAttribute (SgSymbol* var, SgNode* anchor_node);

  //! Return the OpenMP data sharing attribute type of a variable reference
  ROSE_DLL_API omp_construct_enum getDataSharingAttribute (SgVarRefExp* varRef);

...
}

Test/Example code

• https://github.com/rose-compiler/rose/blob/master/tests/nonsmoke/unit/getDataSharingAttribute.C

#include <vector>
#include "OmpSupport.h"  // this file includes several headers declaring OmpSupport namespace
using namespace std;
using namespace OmpSupport;
using namespace SageInterface;

class visitorTraversal : public AstSimpleProcessing
{
  protected:
    virtual void visit(SgNode* n);
};

void visitorTraversal::visit(SgNode* node)
{
  if (SgLocatedNode* lnode = isSgLocatedNode(node))
  {
    //skip system headers
    if (insideSystemHeader (lnode))
      return;

    if (SgForStatement* forloop= isSgForStatement(node))
    {
      cout<<"for loop at line "<< forloop->get_file_info()->get_line() <<endl;
      std::vector< SgVarRefExp * > ref_vec;
      collectVarRefs (forloop, ref_vec);
      for (std::vector< SgVarRefExp * >::iterator iter = ref_vec.begin(); iter!= ref_vec.end(); iter ++)
      {
        SgSymbol* s = (*iter)->get_symbol();
        omp_construct_enum atr = getDataSharingAttribute (*iter);
        // will redirect to a .output file to enable diff-based correctness checking
        cout<<s->get_name()<<"\t"<<toString(atr) <<endl;
      }
    }
  }
}

int main(int argc, char * argv[])
{
  SgProject *project = frontend (argc, argv);

  visitorTraversal myvisitor;
  myvisitor.traverseInputFiles(project,preorder);

  return backend(project);
}

Testing command line:

./getDataSharingAttribute -c -rose:openmp:ast_only openmpcode.c

unparsing of AST nodes

They are defined in

src/backend/unparser/languageIndependenceSupport/unparseLanguageIndependentConstructs.h

src/backend/unparser/languageIndependenceSupport/unparseLanguageIndependentConstructs.C

Enabling OpenMP processing from your translator

Enabling OpenMP parsing and AST creation

The simplest and recommended way is to pass command line option "-rose:OpenMP:ast_only" to your translator.

• This will turn on parsing OpenMP directives and creating dedicated OpenMP AST nodes. Users have the choice of turning on/off of OpenMP parsing.

The 2nd option (recommended) is that you can push the option internally in your translator

// Option 2: recommended 
#include "rose.h"

int main(int argc, char* argv[])
 {
   vector<string> argvList(argv,argv+argc);
  if (your-condition-meet)
   argvList.push_back("-rose:OpenMP:ast_only");

   // the frontend will correctly handle OpenMP.
   // All internal flags will be automatically set correctly.
   // Several internal actions will be turned on in parsing, translation and the connection to the backend compiler
   SgProject* project = frontend (argvList);
   //...
}

The third option (not really recommended), you can replicate all the internally handling in your translator, as shown in https://github.com/rose-compiler/rose/blob/master/src/frontend/SageIII/sage_support/cmdline.cpp and https://github.com/rose-compiler/rose/blob/master/src/frontend/SageIII/sage_support/sage_support.cpp

• this is a bit complex since you are now exposed to both Project and File level internal flags.
• you have to understand details of how several flags work together to control parsing, ast creation, and lowering.
• and explicitly call the OpenMP processing function
• and pass "-DOPENMP" to backend compiler if backend compiler is needed.

// Option 3: not really recommended due to the complexity involved.
#include "rose.h"
#include "ompAstConstruction.h"

int main(int argc, char* argv[])
 {

   vector<string> argvList(argv, argv+argc);

   // the default frontend does not handle OpenMP at all. 
   SgProject* project = frontend (argvList);
  ..
    // optional
    // OmpSupport::enable_debugging = true;

    // We assuming one single input file at a time
    // You need to do this for all file if your translator accepts multiple files
     SgFile * cur_file = project->get_fileList()[0];

   // set three flags related to OpenMP processing
     cur_file->set_openmp(true);  // the top level flag
     cur_file->set_openmp_parse_only(false); 
     cur_file->set_openmp_ast_only(true); 
     cur_file->set_openmp_lowering(false);

    // process OpenMP directives within this file, based on the flag setting
    // At least two phases are involved: one is to do the parsing and AST creation
    // the other is to handle -DOPENMP when connecting to the backend compiler
     OmpSupport::processOpenMP(isSgSourceFile(cur_file));

   AstTests::runAllTests(project);
   return backend(project);
}

Enabling OpenMP Lowering

Sometimes it is desired to call OpenMP lowering from your own translator, not from the builtin frontend() of ROSE. Here is an example:

#include "rose.h"
#include "ompAstConstruction.h"

int main()
 {
  SgProject* project = frontend(argvList,frontendConstantFolding);
  ..
    OmpSupport::enable_accelerator = true;
    OmpSupport::enable_debugging = true;


    // We only process one single input file at a time
     ROSE_ASSERT (project->get_fileList().size() ==1);
     SgFile * cur_file = project->get_fileList()[0];

  // set three flags related to OpenMP processing
     cur_file->set_openmp(true);
     cur_file->set_openmp_lowering(true);
     cur_file->set_openmp_parse_only(false);

     // process OpenMP directives, including omp target
     OmpSupport::processOpenMP(isSgSourceFile(cur_file));

   AstTests::runAllTests(project);
   return backend(project);

}

Instrumentation support

XOMP (ROSE's OpenMP runtime layer) supports a simple event instrumentation to collect timestamps of key OpenMP events. The data collected can be helpful for further analysis, such as investigating serial vs. parallel regions during an execution.

Environment variable to turn on this support

• XOMP_REGION_INSTR =0|1: simply expose this environment variable will turn on this support. This option is turned off by default to avoid unnecessary overhead.
  • in BASH: type "export XOMP_REGION_INSTR=1"

Once this feature is turned on, use rose to compile your OpenMP code. The executable will generate a data file with a time stamp as its prefix, such as "2013_03_05_10_05_45.jacobi". This file will contain a time stamp, value 1 or 2, and file location for each change between serial and parallel execution.

Sample GNU plot script to generate a figure from the data file generated by XOMP

# cat timeline-plot.sc
set xlabel 'time stamps'
set yrange [0:3]
#set output "lulesh.eps" 
#set terminal postscript eps
plot "sp.A.8threads" using ($1-1362509569.298380):2 with lines
#plot "timeline1.data" using 1:2 with lines

Sample generated GNU plot figure: X-axis is the timeline, Y-axis is for the serial (value 1) vs parallel (value 2) execution

Rose-xomp-timeline-npb-mg-A-8threads

,

Rose-xomp-lulesh-timeline-8threads

Relevant commits for this support

• https://github.com/rose-compiler/edg4x-rose/commit/59233fc54208fbb8ad78e24eb76266cec08ee55e , latest
• https://github.com/rose-compiler/edg4x-rose/commit/3f0a87daa6ac24b4cd22fadbef5b4a507317c6bf,
• https://github.com/rose-compiler/edg4x-rose/commit/ff1ab8601365d400f730ea5606f3b4f44316c72a, earliest commit

How large is the implementation?

Data collected on July 7th, 2015

Directly source lines

[liao6@tux322:~/workspace/masterDevClean/sourcetree/src/frontend/SageIII]wc omp*
  1900   6723  75176 ompAstConstruction.cpp
    20     39    504 ompAstConstruction.h
  1504   4821  45545 ompFortranParser.C
   128    365   3361 omp.h
   241   1171   9011 omplexer.ll
   284    779   9714 omp_lib.f90
    57    119   1784 omp_lib.h
    17     76    647 omp_lib_kinds.h
  1056   3010  40607 ompparser.yy
  5207  17103 186349 total

[liao6@tux322:~/workspace/masterDevClean/sourcetree/src/midend/programTransformation/ompLowering]wc *.h *.inc *.cu *.cpp *.sh *.c 
    152     702    7533 libgomp_g.h
     51     208    1814 libompc.h
     57     130    1885 libxompf.h
    387    2424   17562 libxomp.h
    212    1232   10559 omp_lowering.h
    771    2841   38153 run_me_callers2.inc
    771    2841   35845 run_me_callers.inc
  67843  103833 1891978 run_me_defs.inc
  76291  143257 1761968 run_me_task_defs.inc
    973    4246   35237 xomp_cuda_lib.cu
    334    1921   13137 xomp_cuda_lib_inlined.cu
   5939   26563  267555 omp_lowering.cpp
     24     139    1021 run_me_caller_generator2.sh
     21     101     736 run_me_caller_generator.sh
     45     163    1138 run_me_generator.sh
     86     405    2709 run_me_task_generator.sh
    156     584    4418 xomp_accelerator_sched_test.c
    194     821    5919 xomp_accelerator_sched_test_v2.c
   1823    6642   54788 xomp.c
 156130  299053 4153955 total

Indirect support source files

[liao6@tux322:~/workspace/masterDevClean/sourcetree/src/midend/programTransformation/astOutlining]wc *.cc *.cc
   576   1390  15514 ASTtools.cc
   175    539   4869 Block.cc
   307    668   6188 Case.cc
   371   1254  10678 Check.cc
    97    537   3803 CollectVars.cc
    33     79    682 Copy.cc
   101    263   2611 ExtractIfs.cc
   219   1009   9306 GenerateCall.cc
  1422   6803  61202 GenerateFunc.cc
   185    482   3563 If.cc
    63    131   1524 IfDirectiveContextFinder.cc
   163    481   4711 IfDirectiveExtractor.cc
   785   3165  33825 Insert.cc
   149    484   4329 Jumps.cc
    59    120    945 NameGenerator.cc
   292    910   9030 NonLocalControlFlow.cc
    64    176   1895 NonLocalDecls.cc
   373   1299  12924 Outliner.cc
   405   1416  12345 PragmaInterface.cc
    41     94   1191 PrePostTraversal.cc
   102    357   3178 Preprocess.cc
   445   1373  14757 PreprocessingInfo.cc
   436   1352  12210 PreprocIfs.cc
   209    763   7185 StmtRewrite.cc
    33     85    893 This.cc
   293   1045  10043 ThisExprs.cc
   712   3499  33690 Transform.cc
   424   1405  13887 VarSym.cc
   576   1390  15514 ASTtools.cc
   175    539   4869 Block.cc
   307    668   6188 Case.cc
   371   1254  10678 Check.cc
    97    537   3803 CollectVars.cc
    33     79    682 Copy.cc
   101    263   2611 ExtractIfs.cc
   219   1009   9306 GenerateCall.cc
  1422   6803  61202 GenerateFunc.cc
   185    482   3563 If.cc
    63    131   1524 IfDirectiveContextFinder.cc
   163    481   4711 IfDirectiveExtractor.cc
   785   3165  33825 Insert.cc
   149    484   4329 Jumps.cc
    59    120    945 NameGenerator.cc
   292    910   9030 NonLocalControlFlow.cc
    64    176   1895 NonLocalDecls.cc
   373   1299  12924 Outliner.cc
   405   1416  12345 PragmaInterface.cc
    41     94   1191 PrePostTraversal.cc
   102    357   3178 Preprocess.cc
   445   1373  14757 PreprocessingInfo.cc
   436   1352  12210 PreprocIfs.cc
   209    763   7185 StmtRewrite.cc
    33     85    893 This.cc
   293   1045  10043 ThisExprs.cc
   712   3499  33690 Transform.cc
   424   1405  13887 VarSym.cc
 17068  62358 593956 total

[liao6@tux322:~/workspace/masterDevClean/sourcetree/src/frontend/SageIII/sageInterface]wc sageBuilder.C sageBuilder.h sageBuilder_fortran.C sageInterface.C sageInterface.h sageInterface_type.C
  16842   64856  784120 sageBuilder.C
   1489    8428   90956 sageBuilder.h
    104     212    3408 sageBuilder_fortran.C
  19213   71263  847054 sageInterface.C
   2406   14864  128891 sageInterface.h
   2208    7633   87896 sageInterface_type.C
  42262  167256 1942325 total

The OpenMP implementation in ROSE supports OpenMP 3.0 and the accelerator (GPU) support in OpenMP 4.0. It has three components: a frontend, a middle end, and a runtime layer, with 4849, 6151, and 6203 lines of code respectively. In total, there are 17203 lines of code which are dedicated for implementing OpenMP.

In addition, two other supportive components in ROSE are used to help OpenMP implementation: 1) the AST outliner (extracting portions of code into functions) and 2) the AST interface functions (building and manipulating AST). The AST outliner has 17068 lines of code. The AST interface functions have 42262 lines of code. So the lines of indirect source files are 42262.

Counting both direct and indirect source files, there are 76533 lines related to OpenMP implementation in ROSE.

Using an online software construction cost calculator (http://www.csgnetwork.com/costconstrmodelcalc.html) assuming a developer can write and maintain 500 lines of code per month. we estimate that the cost is 48 man-months (or 480K using a 10K monthly cost rate per person) for developing the dedicated OpenMP source files of 17.2K source lines. The total cost for both direct and indirect source files is estimated to be 228 man-months or 2.28 millions.

We have not tried to add the parsing and unparsing components in ROSE into this estimation. The roughly estimation is that there are around 1 million lines of code to parse input code and later unparse the source code as output.

Publications

• Chunhua Liao, Yonghong Yan, Bronis R. de Supinski, Daniel J. Quinlan, and Barbara Chapman. "Early experiences with the openmp accelerator model." In OpenMP in the Era of Low Power Devices and Accelerators, pp. 84-98. Springer Berlin Heidelberg, 2013.
• Chunhua Liao, Daniel J. Quinlan , Thomas Panas and Bronis de Supinski, A ROSE-based OpenMP 3.0 Research Compiler Supporting Multiple Runtime Libraries, the 6th International Workshop on OpenMP (IWOMP), June 14-16, 2010, Tsukuba, Japan. LLNL-CONF-422873