IBM DataPower Gateway - Docker for Local Development
IBM DataPower Gateway is a middleware integration platform, typically used by large corporations and Government organisations, because of its security, stability and performance qualities. Developing for DataPower in a local way can be tricky; because DataPower appliances are generally quite bulky (as physical or VMWare appliances), or difficult run locally (kubernetes) - it’s often useful to run DataPower as a local docker service. A local docker deployment has most of the same capabilities as a traditional production deployment of DataPower, making it a convenient way to accelerate software development.
Best of all, it’s all free (no cost).
Let’s take a look at how to use docker for DataPower.
Setup
We need to have docker installed. Docker is a virtualization technology that allows a thin server to run on your local machine without having a full copy of the Linux kernel, and so on. The offocial docker website has a good guide here.
Docker can be installed on most common consumer operating systems; support is excellent on Linux, Mac, and by even Windows (using Windows Subsystem for Linux). Follow these guides to install the services you need - it’s generally all free.
Running DataPower
Okay, now we want to run a DataPower server. First, we need to pull the DataPower image. An image is basically a copy of the filesystem for the DataPower archetype - it will contain all the IBM DataPower code and configuration that makes it DataPower.
Once upon a time, this was freely available (with registration) on Docker Hub through a command like docker pull ibmcom/datapower:latest
- which would pull the image from Docker Hub. However, the Docker company has changed licensing models and that (plus other perogatives, I am sure) has prompted IBM to move away to their own service. I won’t get into the politics of that - but the net effect is that we need to look elsewhere for images. Note that many older tutorials will still reference this older location.
Instead, per this link, we can now get images from https://www.ibm.com/cloud/container-registry
. The free development version requires no registration or licensing, but the production images do.
Let’s run that (ignore the arguments, I’ll explain that next - and be sure to put all of this on one line or seperate each line with a \ character):
(base) $ docker run -it
-v $PWD/config:/opt/ibm/datapower/drouter/config
-v $PWD/local:/opt/ibm/datapower/drouter/local
-e DATAPOWER_ACCEPT_LICENSE=true
-e DATAPOWER_INTERACTIVE=true
-p 9090:9090
-p 9022:22
-p 5554:5554
-p 8000-8010:8000-8010
--name idg
icr.io/cpopen/datapower/datapower-limited:10.0.1.5
Note - the above is going, at least the first time, download a large amount of content over the internet - you’ll want decent transfer rates. The image is cached locally for future use, though.
If that worked, you’ll see output like:
Unable to find image 'icr.io/cpopen/datapower/datapower-limited:10.0.1.5' locally
10.0.1.5: Pulling from cpopen/datapower/datapower-limited
ad62d8acaeb8: Pull complete
8a4cee2d3973: Pull complete
96000dea3075: Pull complete
e8dad73fb78b: Pull complete
49a4fb41333d: Pull complete
e85a228f8ca1: Pull complete
71ccee499ae2: Pull complete
9b6a1b7553ff: Pull complete
65e01eb1f92b: Pull complete
b6cb10b1308f: Pull complete
3adb595a45c9: Pull complete
2d466df5b123: Pull complete
Digest: sha256:1543db506f66941351a5119ce9b567a3124a6c13158f40a6992b6a9ae9e50010
Status: Downloaded newer image for icr.io/cpopen/datapower/datapower-limited:10.0.1.5
20230502T101959.889Z [0x8040006b][system][notice] logging target(default-log): Logging started.
20230502T101959.936Z [0x804000fe][system][notice] : Container instance UUID: 8be56500-0bf1-4ff9-b860-b526a5a8a465, Cores: 2, vCPUs: 4, CPU model: Intel(R) Core(TM) i3-1005G1 CPU @ 1.20GHz, Memory: 7506.8MB, Platform: docker, OS: dpos, Edition: developers-limited, Up time: 0 minutes
20230502T101959.972Z [0x8040001c][system][notice] : DataPower IDG is on-line.
20230502T101959.973Z [0x8100006f][system][notice] : Executing default startup configuration.
20230502T102000.329Z [0x00350015][mgmt][notice] apiprobe-settings(default): tid(1135): Operational state down
20230502T102000.329Z [0x04f30005][mgmt][error] apic-gw-service(default): tid(1007): The effective gateway peering object is down
20230502T102000.329Z [0x04f30005][cli][error] apic-gw-service(default): The effective gateway peering object is down
20230502T102000.330Z [0x00350015][mgmt][notice] api-security-token-manager(default): tid(1103): Operational state down
20230502T102000.362Z [0x00350014][mgmt][notice] assembly-function(default-func-preflow): tid(6888): Operational state up
20230502T102000.362Z [0x00350014][mgmt][notice] assembly-function-call(default-func-call-preflow): tid(111): Operational state up
20230502T102000.362Z [0x00350014][mgmt][notice] api-rule(default-assembly-rule-global): tid(111): Operational state up
20230502T102000.362Z [0x00350014][mgmt][notice] assembly(default-assembly-global): tid(7000): Operational state up
20230502T102000.362Z [0x00350014][mgmt][notice] assembly-function(default-func-global): tid(7016): Operational state up
20230502T102000.362Z [0x00350014][mgmt][notice] assembly-function-call(default-func-call-global): tid(111): Operational state up
20230502T102000.362Z [0x00350014][mgmt][notice] api-rule(default-assembly-func-rule): tid(111): Operational state up
20230502T102000.364Z [0x8100006d][system][notice] : Executing system configuration.
20230502T102000.365Z [0x8100006b][mgmt][notice] domain(default): tid(7176): Domain operational state is up.
20230502T102657.317Z [0x8040009e][system][notice] throttle(Throttler): tid(1711): Setting throttle thresholds: Memory(-1.000000,-1.000000), Temporary-FS(0.000000,0.000000), XML-Names(0.100000), Timeout(30)
e9106cb9cc7d
Unauthorized access prohibited.
20230502T102657.333Z [0x8040009e][system][notice] throttle(Throttler): tid(1711): Setting throttle thresholds: Memory(-1.000000,-1.000000), Temporary-FS(0.000000,0.000000), XML-Names(0.100000), Timeout(30)
login: 20230502T102657.871Z [0x806000dd][system][notice] cert-monitor(Certificate Monitor): tid(415): Enabling Certificate Monitor to scan once every 1 days for soon to expire certificates
20230502T102657.997Z [0x8100072c][mgmt][warn] quota-enforcement-server(QuotaEnforcementServer): tid(831): A password alias is not defined for the quota enforcement server. The system default is used.
20230502T102732.348Z [0x00350014][mgmt][notice] quota-enforcement-server(QuotaEnforcementServer): tid(831): Operational state up
20230502T102733.243Z [0x8100003b][mgmt][notice] domain(default): Domain configured successfully.
login:
You might need to press enter to get the login prompt.
Arguments
Okay, we passed multiple arguments. Here’s what they do (feel free to change them):
docker run -it # -it runs a terminal and allows input
-v $PWD/config:/opt/ibm/datapower/drouter/config # -v will mount a local directory ($PWD/config) as the DataPower server's config directory - which is where WSP etc config lives, and note that this path has changed for recent DataPower image versions
-v $PWD/local:/opt/ibm/datapower/drouter/local # -v As above, but local directory for general files and code, and also has changed path
-e DATAPOWER_ACCEPT_LICENSE=true # Accept the DataPower license
-e DATAPOWER_INTERACTIVE=true # Allow interactivity with the DataPower CLI
-p 9090:9090 # Map port 9090 on our localhost workstation IP to 9090 on the DataPower server - so we can use the WebGUI once enabled
-p 9022:22 # Map port 9022 to DataPower server's SSH port 22; we use 9022 because lower ports (below 1024 I believe, which are standard 'well-known' ports) require special admin access
-p 5554:5554 # Map port 5554 through for the REST API management; we could likewise do this for the SOMA SOAP admin API
-p 8000-8010:8000-8010 # Map a range or ports through for application purposes
--name idg # Give the docker server we are running a name of idg, just in case we need to identify it specifically
icr.io/cpopen/datapower/datapower-limited:10.0.1.5 # Use the image hosted on this URL - the developer version of 10.0.1.5
Enable the Web UI
At this point, if you have followed the instructions, we have a DataPower server and a login prompt. Let’s login with the default password of admin:admin
:
login: admin
Password: *****
Welcome to IBM DataPower Gateway console configuration.
Copyright IBM Corporation 1999, 2021
Version: IDG.10.0.1.5 build 336212 on Sep 29, 2021 5:46:20 PM
Delivery type: LTS
Serial number: 0000001
idg# configure terminal
Global mode
idg(config)# web-mgmt
Modify Web Management Service configuration
idg(config web-mgmt)# admin-state enabled
idg(config web-mgmt)# exit
Customizing
Okay, so we have seen various arguments. From here, we can do whatever customization we want. We can create, through the UI or SOMA/REST mgmt APIs (when enabled) whatever configuration we want, provided we use port 8000 to 8010 per the arguments above. Any updates we make and save, should update the $PWD/config and /local directories when we use the save config
CLI or UI command.
So, one way of using this is to apply local configuration changes to the DataPower server and take whatever local and config changes you want, add to source control - and then publish to a real server where you might do testing and integration. That’s a pretty good use of this development edition.
It’s also perfect for running staging and unit testing - you might provision a docker server, deploy some config, run some SoapUI or Postman tests, and then tear down the server.
Finally - for Production, you absolutely can use a docker deployment - using a production, supported image. This provides a basically identical deployment for local development. Similarly, for kubernetes containerized environments, including openshift, one can add file layers (basically, extra files) on top of a clean DataPower image for a 0-runtime-extra-config approach to immediately booting up a DataPower server with all the config you need.
Cleaning Up
After you’re done using the server, you can kill it by exiting out of the terminal process as per normal, and then remove it with docker remove idg
where idg is the name we gave it before. This will clean up the server that we created, but won’t clear the volumes we gave it with -v
; so rerunning the container create command we first used should spin up the server again with the same DataPower config (WSPs etc) that we had created and saved before.
docker ps
will give us a list of docker containers on this machine. There are, in fact, a bunch of docker commands that make working with docker containers easy.
Conclusion
We’ve seen here how to setup Docker and install a DataPower service upon it, that can be freely used for local development.
Until next time! Chris