A quick hack on Java code formatter

Some time ago, I got a requirement to have some Java code generated and formatted, also, have a bunch of old code formatted.

Since former requires formatting being done in Java library, I am not able to utilize the feature in IDE like IntelliJ IDEA.

The ideal solution would be a Java library that formats Java code.

After some search on Internet, there appears not many libraries available. The closest one would be google-java-format. However it is a rather opinionated solution, and sadly its opinion is not compatible with my requirement.

So far, bad news. The good news is that everything happens in Open Source world. This means I could start hack without concern too much on license. Let us fork!

Quick forward a few hours, I get the job done. A customized google-java-format.

Oh, another bad news. google-java-format 1.8 is based on Java 11. It relies on com.sun.tools.javac.parser.JavacParser to parse code to Java AST (abstract syntax tree). And my target Java library is based on Java 8.

OK, one more hack, reflection. If runtime is Java 11 or higher, call code formatter, else skip it.

I know, the whole business is not elegant. But it get the job done, and quick. Open Source is the productivity multiplier here.

PS, this blog on dartfmt prevented me to try to write a new code formatter from Java AST.

Posted in Computer and Internet, Programming and Algorithm | Tagged | Leave a comment

Web app as proxy to manage Azure resource

With more of my cloud resources resides on Azure, the necessity arises that I be able to administrate some cloud resources without too much trouble on dedicated toolchain and/or authentication.

A quick solution is to create a web app as proxy to manage other cloud resources. By sending simple HTTPS request, one can do pre-configured tasks like power on/off a virtual machine.

Of course, the first consideration is security. The solution ranges from OAuth to Basic to simple secret. At present since the management is non-critical, I just use a simple secret. Later I will find time to switch to OpenID.

My previous post on Azure pipeline + Azure web app detailed the guidance on how to create Azure web app, and use Azure pipeline to automate the deployment.

To enable the web app to manage Azure resources, one additional configure is required. Web app needs to turn on managed identity, and that identity needs to be assigned necessary access control for the management of the cloud resources.

If web app is created via Azure Java management SDK, 2 additional line would suffice. Sample code here (line 123 should be modified to suit the requirement).

If web app is created via Portal, things get a bit confused. Managed identity is enabled in web app identity.

Managed identity (system assigned)

Access control is instead assigned in the resource to be managed by the web app.

Access control (managed identity of az-mgmt has access control to vmess resource group)

With managed identity and access control all set, one only need a few lines of Azure Java management SDK code to handle the authorization to Azure management in web app. Sample code here.

        client = Azure.configure().withLogLevel(LogLevel.BASIC)
                .authenticate(new AppServiceMSICredentials(AzureEnvironment.AZURE))
                .withSubscription(subscriptionId);

After management client authenticated, the web app can use the client to do any Azure management within the access control. Sample code to power on/off of virtual machine here.

In summary, I deploy a web app with Azure management features, to substitute a simple but limited REST API for the complete but unhandy Portal UI. Whether the trade-off worths the effort, well, is indeed a subjective matter.

Posted in Cloud, Computer and Internet | Tagged | Leave a comment

Darwin Among the Machines

Darwin Among the Machines by George Dyson.

This is my second time reading through the book. It is a work trying to ask right questions, not giving correct answers.

First chapter Leviathan is already dense and inspiring, filled with subtle but intriguing ideas about logic and compute, intelligence at scales below and above us, network and emergence, symbiosis and game theory. This chapter is indeed a miniature of the whole book.

Try to think, when no single human being would now ever grasp the entirety of our knowledge (or even a small part of it), what is the definition of intelligence, and how it would ever work on society at level well above each individuals? And then, in our time, how human society would coexist with computer and network, which not only calculate/model/predict faster then us, but also evolve faster than us? Moreover, what happens when a distributed intelligence no longer be content with modelling on what we let it to model, but also evolve the ability and willingness to model the behavior of itself. Is it now conscious?

Too many unknowns. The point is, ask the right question, and don’t stop there.

PS:

There is certain book that I do not know how to talk about, even with multiple reads, i.e., The Denial of Death by Ernest Becker.

Just read it.

Posted in Book, Computer and Internet, Science | Tagged , | Leave a comment

Rise of the Robots

Rise of the Robots: Technology and the Threat of a Jobless Future by Martin Ford.

It has been a while since I read the book. Definitely it was a good read.

Historically, technological innovation obsoleted jobs that easily being automated, meanwhile also created new jobs in new areas, usually with better working condition. In the process, technology improved productivity and raise the living standard of general public.

Now the question is, with AI revolution in sight, will it remains the same process?

Martin Ford argued again it. And he backed the argument with serious economic data.

The balance of capital and labour was tipping against labour. This is already happening, and it does not require a singularity scenario. The current advance on narrow AI, and the future development of modularity, mass-production and cost-reduction on robotics, would be enough to displace labour on routine job, or at least continue at keeping the wage low.

Higher education might not help much, since most job in agriculture, industry, and many in service, is routine and predictable, which is susceptible for automation.

Education and health-care is likely to be the area which being more resistant to automation. Hence the cost would remain high.

With wage kept low, but education and health-care cost stays high (as well as property price), the purchase power of average worker would remain low.

The low income, low expenditure loop could become a vicious positive-feedback that drives economics down, if not properly countered.

This is an existing problem for developed countries like U.S.A., and it would be a more serious problem for developing countries, which is still in the process of being industrialized.

Martin Ford put the universal income as an important part of the solution to the problem.

Posted in Book | Tagged | Leave a comment

edX course – Masterpieces of World Literature – and it’s recommended reading

Masterpieces of World Literature walks you through history, landscape, and imagination — sampling and refining from a large corpus of world literature.

A pity I haven’t finished the reading material before lossing access to the course. Hence here I would record the list.

Below all quoted from the course material.


The Epic of Gilgamesh

The course team recommends the English translation by Andrew George. For a public domain English translation, you can use R. Campbell Thompson’s version, and you can find a version of some of the Babylonian fragments on Project Gutenberg.

The Odyssey

The advised reading for this module includes books 5-12 of the Odyssey. Strongly encouraged but not required are the first book and books 23 and 24. The course team recommends the English translation by Robert Fagles. But you can find a public domain English translation at the Perseus Project. If you would like to look at an edition of The Odyssey in the original Greek, you can also find that at the Perseus Project.

The 1001 Nights

Advised readings include the Prologue, The Story of King Shahrayar and Shahrazad, His Vizier’s Daughter, The Tale of the Ox and the Donkey, The Tale of the Merchant and His Wife, The Story of the Merchant and the Demon, The First Old Man’s Tale, The Second Old Man’s Tale, The Third Old Man’s Tale, The Story of the Fisherman and the Demon, The Tale of King Yunan and the Sage Duban, The Tale of the Husband and the Parrot, The Tale of the King’s Son and the She–Ghoul, The Tale of the Enchanted King.

For a modern English translation, the course team recommends the The Arabian Nights, trans. Husain Haddawy (Norton, 2008). The 1885 translation of the 1001 Nights by the British explorer and Arabist Richard Francis Burton is available via the Internet Archive.

The Tale of Genji

The advised reading assignment includes: From Chapter I. The Lady of the Paulownia–Courtyard Chambers; From Chapter II. Broom Cypress; From Chapter V. Little Purple Gromwell; From Chapter VII. An Imperial Celebration of Autumn Foliage; From Chapter IX. Leaves of Wild Ginger; From Chapter XII. Exile to Suma; From Chapter XIII. The Lady at Akashi; From Chapter XXV. Fireflies; From Chapter XL. The Rites.

For an English translation, the course team recommends Royall Tyler’s translation. You may find a public domain English translation of the text by Edward G. Seidensticker at the University of Oxford Text Archive. If you would like to attempt to read a 1654 Japanese version of the text, you can find it at the Asian Division of the Library of Congress.

The Lusiad

The advised reading for this module is the entirety of The Lusiads. You can read the epic in English translation at Project Gutenberg. You may also find it in the original Portuguese at oslusiadas.org/i/.

Candide

Your advised reading is the entirety of Candide. Here is a link to the 1918 English edition of Candide translated by William F. Fleming hosted by Project Gutenberg. Multiple editions in French are available online. The original 1759 edition is available from the Bibliothèque nationale de France. The 1829 Beuchot edition of the text is viewable via Wikisource.

Lu Xun and Eileen Chang

The advised readings for this week include the full text of Lu Xun’s "Diary of a Madman" and Eileen Chang’s "Sealed Off." The course team recommends the English translation of Lu Xun’s work by William A. Lyell. You can find a public domain English translation of "Diary of a Madman" here, and in the original Chinese at Project Gutenberg. You can find Chang’s "Sealed Off" in English translation in Love in a Fallen City, but the text is also published online with permission of New York Review Books.

Borges

The advised reading this week is Borges’ story, "The Garden of Forking Paths." We also strongly recommend "Pierre Menard, Author of the Quixote," "The Library of Babel," "Tlön Uqbar, Orbis Tertius," and "Death and the Compass." The course team recommends the English translations by Anthony Kerrigan, in the volume Ficciones. You may find some of Borges’ works in the original Spanish at Open Library.

Death and the King’s Horseman

The recommended reading for the week is Death and the King’s Horseman in its entirety.

Salman Rushdie and Jhumpa Lahiri

The recommended readings include: From Rushdie’s East, West, “The Prophet’s Hair,” “At the Auction of the Ruby Slippers,” and if time permits, all three of the final set of stories, but particularly “Chekov and Zulu.” From Lahiri’s Interpreter of Maladies: "A Temporary Matter," "Interpreter of Maladies," and "The Third and Final Continent."

Orhan Pamuk

The course team recommends you read My Name Is Red. Most important are the first 7 very short chapters, chapters 10, 12, 13, 14, 16, as well as chapters 18, 19, and 20. This way you will get a sense of the main characters and of the main conflict. We also recommend chapter 29, which contains a detailed description of miniatures, and chapter 47. But if you’d like to know how the novel ends, you should keep reading!

Posted in Book | Tagged | Leave a comment

TEAC P90SD 更换电池

用了4年的P90SD,电池终于死的不能再死了。TB上买了安桥HA300的兼容电池(两机型基本就是运放不同)。

参考使用说明书和这篇拆解

  1. 侧边外壳是4颗1.3mm的六角螺丝。
  2. OLED上的玻璃是胶水粘的。需要加热,或者真空吸盘+翘板。
  3. 兼容电池比原装略厚。

最终成功复活。然而我再不想折腾第二次了…

Posted in Entertainment | Leave a comment

SpringBoot + GitHub + Azure Pipeline + Azure Web App

I still admire the ease to deploy Heroku with Java. But since I got some credit on Azure, let me try to make a similar workflow.

I would assume one have an Azure account.

First step, as usual, create a SpringBoot web app. Note the final target might need to be named as app.jar to be picked by Azure without configure.

One can put the code on GitHub or BitBucket or Azure Repos (I think it comes with Azure DevOps) or else.

I wrote a simple webapp to auto-detect charset of uploaded text file and convert to UTF-8, no database yet.

Second step, on Azure, create Azure Web App. Choose Java SE on Linux platform (ongoing support for Windows). There is a free tier F1 SKU (it is slow, as expected, but free of charge).

This step could also be done as Java code (infrastructure as code) via Azure CLI or Java (among other options).

Third step, create Azure Pipeline to build and deploy. It connects the code from one’s repository and webapp deployment on Azure. One can enter Azure DevOps from either GitHub Marketplace or Azure.

Create a new Builds Pipeline, select the repository. DevOps would automatically detect the build tool. Choose “Maven package Java project Web App to Linux on Azure” (I would assume similar for Gradle).

Connect with Azure and choose the Web App name created in second step. Review the final YAML configure (it should contains 3 jobs in Build stage, and 1 job in Deploy stage).

And it is done. First build/deploy would be ongoing, and next time code change on master new build/deploy would run automatically.

Code committed to repository would be live in a few minutes.

PS, a few side notes:

  1. tika-core has the interface, but to detect charset, tika-parsers is required.
  2. There is still bugs in Azure Pipeline (like status not updated), hence there is comment trigger.
Posted in Cloud, Computer and Internet | Tagged | 1 Comment

service discovery in kubernetes

Kubernetes utilizes DNS for service discovery. It is quite easy. One can use host name service_name.namespace (in my previous sample, code-sharing-api.default) to access the service (with load-balancing).

I was preparing a sample front-end webapp, and then figured out that it actually required an API gateway (Ingress, in kubernetes), instead of service discovery mechanism. Else I would face the CORS problem (front-end and back-end being on different host or port), unnecessarily.

Posted in Cloud, Computer and Internet | Tagged | Leave a comment

spring boot, docker, kubernetes

In previous post, I’ve started a working Minikube on laptop. But the deployment used a prebuilt Node.js echo server image.

Here I will go through webapp built by Spring Boot, Docker image, and Kubernetes deployment/service configured via YAML.

First, make a simple webapp by Spring Boot. Any app would do. Here I just paste my sample app.

<!--pre class="wp-block-syntaxhighlighter-code brush: groovy; notranslate"-->plugins {
	id 'org.springframework.boot' version '2.1.6.RELEASE'
	id 'java'
	id 'io.freefair.lombok' version '3.8.0'
}

apply plugin: 'io.spring.dependency-management'

group = 'org.weidongxu.app'
version = '1.0.0'
sourceCompatibility = '1.8'

repositories {
	mavenCentral()
}

dependencies {
	implementation 'org.springframework.boot:spring-boot-starter-web'
	testImplementation 'org.springframework.boot:spring-boot-starter-test'
}<!--/pre-->
<!--pre class="wp-block-syntaxhighlighter-code brush: java; notranslate"-->package org.weidongxu.app.codesharingapi;

import lombok.Getter;
import lombok.Setter;
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.http.MediaType;
import org.springframework.web.bind.annotation.*;

import javax.annotation.PostConstruct;
import java.time.Instant;
import java.util.ArrayList;
import java.util.List;
import java.util.UUID;
import java.util.concurrent.ConcurrentHashMap;
import java.util.concurrent.ConcurrentMap;

@SpringBootApplication
@RestController
public class CodeSharingApiApplication {

	public static void main(String[] args) {
		SpringApplication.run(CodeSharingApiApplication.class, args);
	}

	@GetMapping(
			value = "/",
			produces = MediaType.APPLICATION_JSON_VALUE
	)
	public List<CodeBlock> get() {
		List<CodeBlock> blocks = new ArrayList<>(codeBlocks.values());
		blocks.sort((CodeBlock o1, CodeBlock o2) -> Long.compare(o1.getModifiedTsMs(), o2.getModifiedTsMs()));
		return blocks;
	}

	@PostMapping(
			value = "/",
			consumes = MediaType.APPLICATION_JSON_VALUE,
			produces = MediaType.APPLICATION_JSON_VALUE
	)
	public CodeBlock post(@RequestBody CodeBlock codeBlock) {
		codeBlocks.put(codeBlock.getId(), codeBlock);
		return codeBlock;
	}

	@PostConstruct
	private void postConstruct() {
		CodeBlock codeBlock = new CodeBlock("print [0..]");
		codeBlocks.put(codeBlock.getId(), codeBlock);
	}

	private final ConcurrentMap<String, CodeBlock> codeBlocks = new ConcurrentHashMap<>();

	private static class CodeBlock {
		@Getter @Setter private String id;
		@Getter @Setter private String code;
		@Getter @Setter private long version;
		@Getter @Setter private long modifiedTsMs;

		public CodeBlock() {
			this.id = UUID.randomUUID().toString();
			this.version = 1L;
			this.modifiedTsMs = Instant.now().toEpochMilli();
			this.code = "";
		}

		public CodeBlock(String code) {
			this();
			this.code = code;
		}

		public CodeBlock(String id, String code, long version, long modifiedTsMs) {
			this.id = id;
			this.code = code;
			this.version = version;
			this.modifiedTsMs = modifiedTsMs;
		}
	}
}<!--/pre-->

Build it.

./gradlew build

Note Spring Boot uses port 8080 by default (it can be modified in application.properties).

Second, create Docker image for the webapp. Make sure docker command line is installed.

I will reuse docker daemon within Minikube VM. Before creating Docker image, start Minikube and source its docker env.

minikube start --image-mirror-country cn
eval $(minikube docker-env)

Create a Dockerfile for the webapp.

FROM openjdk:8-jre
VOLUME /tmp
COPY build/libs/*.jar app.jar
ENTRYPOINT ["java", "-jar", "/app.jar"]

Build Docker image.

docker build -t org.weidongxu.app/code-sharing-api:v1 .

Finally, create deployment and service on Kubernetes. This time I use YAML configuration file.

<!--pre class="wp-block-syntaxhighlighter-code brush: plain; notranslate"-->apiVersion: apps/v1
kind: Deployment
metadata:
  name: code-sharing-api
  labels:
    app: code-sharing-api
spec:
  replicas: 2
  selector:
    matchLabels:
      app: code-sharing-api
  template:
    metadata:
      labels:
        app: code-sharing-api
    spec:
      containers:
      - name: code-sharing-api
        image: org.weidongxu.app/code-sharing-api:v1
        imagePullPolicy: Never
        ports:
        - containerPort: 8080<!--/pre-->

Note that I am using docker daemon inside Minikube VM, hence imagePullPolicy: Never.

<!--pre class="wp-block-syntaxhighlighter-code brush: plain; notranslate"-->apiVersion: v1
kind: Service
metadata:
  name: code-sharing-api
  labels:
    app: code-sharing-api
spec:
  type: NodePort
  selector:
    app: code-sharing-api
  ports:
  - protocol: TCP
    nodePort: 30001
    port: 8080
    targetPort: 8080<!--/pre-->

And call kubecrl to create the deployment and service.

kubectl create -f k8s-deployment.yaml
kubectl create -f k8s-service.yaml
Posted in Cloud, Computer and Internet | Tagged | Leave a comment

taste some kubernetes, start with minikube

I was not a big fan on container technology. Well, duty calls. So I would start with tutorials and Minikube on laptop.

Installation is simple. Instruction is here.

One need kubectl and minikube, both is available as pre-built binary on Mac, and can be readily downloaded via curl.
For simplicity best not to move them to /usr/local/bin as in instruction. Just put them somewhere in user PATH would do.

One more requirement is hypervisor. I would use VirtualBox for Mac (HyperKit need a bit work, while VMWare Fusion cost money).

After installation, one need to start Minikube for the first-time. minikube start would do.

However, for mainland China, one had to retrieve ISO and image from AlibabaCloud mirror. Command line would be

minikube start --iso-url 'https://kubernetes.oss-cn-hangzhou.aliyuncs.com/minikube/iso/minikube-v1.2.0.iso' --image-mirror-country cn

for current Minikube version 1.2.0. There is a GitHub with modified default ISO URL. Here the ISO URL could be found as DefaultISOURL at minikube/pkg/minikube/constants/constants.go
If there is future upgrade, remember to use latest version.

After Minikube is started, one can try with some deployment. Start a Node.js echo server (again, mainland China need to use a mirror, instead of default gcr.io).

kubectl create deployment hello-node --image=registry.cn-hangzhou.aliyuncs.com/google_containers/echoserver:1.10

It will create echo server deployment on the pod. The server can be accessed locally within the pod.

kubectl exec $POD_NAME -- curl localhost:8080

Next, one can expose it as service.

kubectl expose deployment hello-node --type=LoadBalancer --port=8080

Check status.

kubectl get all

Sample response (note the external NODE_PORT on service/hello-node, 30252).

NAME                              READY   STATUS    RESTARTS   AGE
 pod/hello-node-67c8cf8784-9t2h6   1/1     Running   0          15m
 

 

 NAME                 TYPE           CLUSTER-IP     EXTERNAL-IP   PORT(S)          AGE
 service/hello-node   LoadBalancer   10.104.130.7   <pending>     8080:30252/TCP   3m49s
 service/kubernetes   ClusterIP      10.96.0.1      <none>        443/TCP          5h6m
 

 

 NAME                         READY   UP-TO-DATE   AVAILABLE   AGE
 deployment.apps/hello-node   1/1     1            1           16m
 

 NAME                                    DESIRED   CURRENT   READY   AGE
 replicaset.apps/hello-node-67c8cf8784   1         1         1       15m
 

All is fine. Now it can be accessed externally.

minikube service hello-node

Above command line is just a convenience, simple

curl $(minikube ip):$NODE_PORT

would do.

Further, deployment can be scaled-up (to 2 pods).

kubectl scale deployment.apps/hello-node --replicas=2

After having enough of our echo service, one can shut it down.

kubectl delete service hello-node
kubectl delete deployment hello-node

Further, stop Minikube (to save some CPU and memory).

minikube stop
Posted in Cloud, Computer and Internet | Tagged | Leave a comment