The Ampere Porting Advisor Tutorial — SitePoint

The Ampere Porting Advisor is a fork of the Porting Advisor for Graviton, an open supply undertaking from AWS, which, in flip, is a fork of the Arm Excessive Efficiency Computing group’s Porting advisor.

Initially, it was coded as a Python module that analyzed recognized incompatibilities for C and Fortran code. This tutorial walks you thru constructing and utilizing the instrument and tips on how to act on points recognized by the instrument.

The Ampere Porting Advisor is a command line instrument that analyzes supply code for recognized code patterns and dependency libraries. It then generates a report with any incompatibilities with Ampere’s processors. This instrument offers options of minimal required and/or advisable variations to run on Ampere processors for each language runtime and dependency libraries.

It may be run on non-Arm64 based mostly machines (like Intel and AMD) and Ampere processors will not be required. This instrument doesn’t work on binaries, solely supply code. It doesn’t make any code modifications, it doesn’t make API degree suggestions, nor does it ship knowledge again to Ampere.

Please Be aware: Though we do our greatest to seek out recognized incompatibilities, we nonetheless suggest to carry out the suitable exams to your software on a system based mostly on Ampere processors earlier than going to manufacturing.

This instrument scans all information in a supply tree, no matter whether or not they’re included by the construct system or not. As such, it could erroneously report points in information that seem within the supply tree however are excluded by the construct system. At present, the instrument helps the next languages/dependencies:

Python 3+

• Python model
• PIP model
• Dependency variations in necessities.txt file

Java 8+

• Java model
• Dependency variations in pom.xml file
• JAR scanning for native technique calls (requires JAVA to be put in)

Go 1.11+

• Go model
• Dependency variations on go.mod file

C, C++, Fortran

• Inline meeting with no corresponding aarch64 inline meeting.
• Meeting supply information with no corresponding aarch64 meeting supply information.
• Lacking aarch64 structure detection in autoconf config.guess scripts.
• Linking in opposition to libraries that aren’t obtainable on the aarch64 structure.
• Use of structure particular intrinsic.
• Preprocessor errors that set off when compiling on aarch64.
• Use of outdated Visible C++ runtime (Home windows particular).
• The next varieties of points are detected, however not reported by default:
  • Compiler particular code guarded by compiler particular pre-defined macros.
• The next varieties of cross-compile particular points are detected, however not reported by default.
  • Structure detection that is dependent upon the host reasonably than the goal.
  • Use of construct artifacts within the construct course of.

For extra data on tips on how to modify points reported, use the instrument’s built-in assist: ./porting-advisor-linux-x86_64 -–assist

Should you run into any points, see our CONTRIBUTING file in the undertaking’s GitHub repository.

Working the Ampere Porting Advisor as a Container

By utilizing this selection, you don’t want to fret about Python or Java variations, or some other dependency that the instrument wants. That is the quickest technique to get began.

Pr-requisites

• Docker or containerd + nerdctl + buildkit

Run Container Picture

After constructing the picture, we will run the instrument as a container. We use -v to mount a quantity from our host machine to the container.

We are able to run it on to console:

docker run --rm -v my/repo/path:/repo porting-advisor /repo

Or generate a report:

docker run --rm -v my/repo/path:/repo -v my/output:/output porting-advisor /repo --output /output/report.html

Home windows instance:

docker run --rm -v /c/Customers/myuser/repo:/repo -v /c/Customers/myuser/output:/output porting-advisor /repo --output /output/report.html

Working the Ampere Porting Advisor as a Python Script

Pr-requisites

• Python 3.10 or above (with PIP3 and venv module put in).
• (Optionally) Open JDK 17 (or above) and Maven 3.5 (or above) if you wish to scan JAR information for native strategies.
• Unzip and jq is required to run take a look at instances.

Allow Python Setting

Linux/Mac:

python3 -m venv .venv supply .venv/bin/activate

Powershell:

python -m venv .venv ..venvScriptsActivate.ps1

Set up necessities

• pip3 set up -r necessities.txt

Run instrument (console output)

• python3 src/porting-advisor.py ~/my/path/to/my/repo

Run instrument (HTML report)

• python3 src/porting-advisor.py ~/my/path/to/my/repo –output report.html

Working the Ampere Porting Advisor as a Binary

Producing the Binary

Pre-requisites

• Python 3.10 or above (with PIP3 and venv module put in).
• (Optionally) Open JDK 17 (or above) and Maven 3.5 (or above) if you’d like the binary to have the ability to scan JAR information for native strategies.

The construct.sh script will generate a self-contained binary (for Linux/MacOS). Will probably be output to a folder referred to as dist.

By default, it would generate a binary named like porting-advisor-linux-x86_64. You possibly can customise generated filename by setting atmosphere variable FILE_NAME.

For Home windows, the Construct.ps1 will generate a folder with an EXE and all of the information it requires to run.

Working the Binary

Pre-requisites

After getting the binary generated, it would solely require Java 11 Runtime (or above) if you wish to scan JAR information for native strategies. In any other case, the file is self-contained and doesn’t want Python to run.

Default habits, console output:

$ ./porting-advisor-linux-x86_64 ~/my/path/to/my/repo

Producing HTML report:

$ ./porting-advisor-linux-x86_64 ~/my/path/to/my/repo --output report.html

Producing a report of simply dependencies (this creates an Excel file with simply the dependencies we discovered on the repo, no options supplied):

$ ./porting-advisor-linux-x86_64 ~/my/path/to/my/repo --output dependencies.xlsx --output-format dependencies

Understanding an Ampere Porting Advisor Report

Right here is an instance of the output report generated with a pattern undertaking:

./dist/porting-advisor-linux-x86_64 ./sample-projects/ 
| Elapsed Time: 0:00:03 
 
Porting Advisor for Ampere Processor v1.0.0 
Report date: 2023-05-10 11:31:52 
 
13 information scanned. 
detected go code. min model 1.16 is required. model 1.18 or above is advisable. we detected that you've model 1.19. see https://github.com/AmpereComputing/ampere-porting-advisor/blob/predominant/doc/golang.md for extra particulars. 
detected python code. in the event you want pip, model 19.3 or above is advisable. we detected that you've model 22.3.1 
detected python code. min model 3.7.5 is required. we detected that you've model 3.10.9. see https://github.com/AmpereComputing/ampere-porting-advisor/blob/predominant/doc/python.md for extra particulars. 
./sample-projects/java-samples/pom.xml: dependency library: leveldbjni-all just isn't supported on Ampere processor. 
./sample-projects/java-samples/pom.xml: utilizing dependency library snappy-java model 1.1.3. improve to no less than model 1.1.4 
./sample-projects/java-samples/pom.xml: utilizing dependency library zstd-jni model 1.1.0. improve to no less than model 1.2.0 
./sample-projects/python-samples/incompatible/necessities.txt:3: utilizing dependency library OpenBLAS model 0.3.16. improve to no less than model 0.3.17 
detected go code. min model 1.16 is required. model 1.18 or above is advisable. we detected that you've model 1.19. see https://github.com/AmpereComputing/ampere-porting-advisor/blob/predominant/doc/golang.md for extra particulars. 
./sample-projects/java-samples/pom.xml: utilizing dependency library hadoop-lzo. this library requires a guide construct  extra information at: https://github.com/AmpereComputing/ampere-porting-advisor/blob/predominant/doc/java.md#building-jar-libraries-manually 
./sample-projects/python-samples/incompatible/necessities.txt:5: dependency library NumPy is current. min model 1.19.0 is required. 
detected java code. min model 8 is required. model 17 or above is advisable. see https://github.com/AmpereComputing/ampere-porting-advisor/blob/predominant/doc/java.md for extra particulars. 
 
Use --output FILENAME.html to generate an HTML report. 

• Within the report, we see a number of language runtimes (Python, pip, golang, Java) and their variations detected. All these messages talk the minimal model and advisable model for these languages. A few of these strains detect that prerequisite variations have been discovered and are purely informative.
• We additionally see some messages from the dependencies detected within the Challenge Object Mannequin (POM) or a Java undertaking. These are dependencies that shall be downloaded and used as a part of a Maven construct course of, and we see three varieties of actionable messages:

Dependency requires more moderen model

./sample-projects/java-samples/pom.xml: utilizing dependency library snappy-java model 1.1.3. improve to no less than model 1.1.4

• Messages of this sort point out that we should always use a more moderen model of the dependency, which would require rebuilding and validation of the undertaking earlier than persevering with .

Dependency requires a guide construct

./sample-projects/java-samples/pom.xml: utilizing dependency library hadoop-lzo. this library requires a guide construct extra information at: https://github.com/AmpereComputing/ampere-porting-advisor/blob/predominant/doc/java.md#building-jar-libraries-manually

• On this case, a dependency does assist the structure, however for some motive (maybe to check {hardware} options obtainable and construct an optimized model of the undertaking for the goal platform) the undertaking have to be manually rebuilt reasonably than counting on a pre-existing binary artifact

Dependency just isn’t obtainable on this structure

./sample-projects/java-samples/pom.xml: dependency library: leveldbjni-all just isn’t supported on Ampere processor.

• On this case, the undertaking is specified as a dependency however just isn’t obtainable for the Ampere platform. An engineer might have to look at what’s concerned in making the code from the dependency compile accurately on the goal platform. This course of will be easy however may take appreciable effort and time. Alternatively, you may adapt your undertaking to make use of an alternate bundle offering related performance which does assist the Ampere structure and modify your undertaking’s code appropriately to make use of this different.

A Transition Instance for C/C++

MEGAHIT is an NGS assembler instrument obtainable as a binary for x86_64. A buyer needed to run MEGAHIT on Arm64 as a part of an structure transition. However the compilation failed on Arm64 within the first file:

The developer needed to know what wanted to be modified to make MEGAHIT compile accurately on Arm64.

On this case, Ampere Porting Advisor (APA) can play a key function. After scanning the supply repository of the MEGAHIT undertaking with APA, we get an inventory of points that have to be checked earlier than rebuilding MEGAHIT on Arm64:

Let’s examine every error kind within the record and proper them for Arm64 if obligatory.

Structure-specific construct choices

These errors shall be triggered as soon as APA detected construct choices not legitimate on Arm64.

The unique CMakeList.txt is utilizing x86_64 compile flags by default with out checking CPU Architectures. To repair this, we will take a look at a CMAKE_SYSTEM_PROCESSOR situation to verify the flags reported by APA shall be solely utilized to x86_64 architectures.

Structure particular directions

The structure particular directions error shall be triggered as soon as APA detected non-Arm64 C-style features getting used within the code. Intrinsic directions are compiled by the compiler instantly into platform-specific meeting code, and sometimes every platform could have their very own set of intrinsics and meeting code directions optimized for that platform.

On this case, we will make the usage of pre-processor conditionals to solely compile the _pdep_u32/64 and __cpuid/ex directions when #if outlined(x86_64) is true for the HasPopcnt() and HasBmi2() features. For vec_vsx_ld, it’s already wrapped in a pre-processor conditional, and can solely be compiled on Energy PC structure, so we will depart it as is.

Structure particular inline meeting

The structure particular directions error shall be triggered as soon as APA detected meeting code getting used within the code. We have to verify whether or not the snippet of meeting code is for Arm64 or not.

The MEGAHIT undertaking solely makes use of the bswap meeting code in phmap_bits.h when it’s being compiled on the x86_64 structure. When being compiled on different architectures, it compiles a fall-back implementation from glibc. So no modifications are required in phmap_bits.h.

In cpu_dispatch.h,two inline features HasPopcnt() and HasBmi2() unconditionally embrace the x86_64 meeting instruction cpuid to check for CPU options on x86_64. We are able to add a precompiler conditional flag #if outlined(x86_64) to verify this code just isn’t referred to as on Arm64, and we are going to at all times return false.

Structure particular SIMD intrinsic

The structure particular directions error shall be triggered as soon as APA detected x86_64 SIMD directions like AVX256 or AVX512 getting used within the code. These SIMD directions are wrapped by precompiler conditional flags and can normally not trigger any performance challenge on Arm64.

If there have been no SIMD implementation of the algorithm for Arm64, there could possibly be a efficiency hole in comparison with x86_64. On this case, there’s a NEON SIMD implementation for Arm64 in xxh3.h and this implementation shall be cherry picked by the compiler based mostly on the CPU structure. No additional actions have to be taken.

Preprocessor error on AArch64

The preprocessor error shall be triggered by APA to point that the Arm64 structure might not be included in a pre-compile stage. On this case, we will see that the pre-compile conditional is for x86_64 solely and doesn’t concern the Arm64 structure.

Rebuild and take a look at

As soon as all these changes have been made, we may rebuild the undertaking:

The undertaking compiled efficiently. We then checked whether or not it handed the undertaking’s take a look at suite:

After we now have manually checked and glued all of the potential pitfalls reported by APA, MEGAHIT is now in a position to construct and run on Ampere processors.