DittoTRX: A Maltego Transform Server for IDN Homograph Attacks

The DittoTRX data integration is developed by Philipp Mieden, who currently studies MS, Security and Network Engineering in the University of Amsterdam and participates in Maltego’s Academic and Non-Profit Program since 2020. This article is originally written and published on his blog.

A while ago, Simone (@evilsocket) published his awesome ditto tool for checking a domain for active imposter attacks, and I finally found some time to play with it.

In this blog post, I’ll share some insights from initial experiments and recon, and open source the toolchain I created to run a central ditto Transform server to analyze the data in Maltego.

With the ditto Transform server, multiple analysts can run Transforms to check domains for active imposters without calculating the variations and checking them on their own machine, but on a dedicated server instead. The server can take care of throttling the external lookups to avoid being rate limited. In addition, future updates could add a queuing mechanism.

Releasing DittoTRX Transforms for Maltego 🔗︎

This 2-part article series will walk you through the release of the ditto Transform server, setting up the server, and how to use the Transforms in Maltego.

Part 1: Setting Up DittoTRX for Maltego
- Intro to IDN Homograph Attacks
- Maltego Codes
Part 2: Using DittoTRX Transforms in Maltego

READ PART 2

IDN Homograph Attacks 🔗︎

Internationalized domain name (IDN) homograph attacks exploit the fact that characters from different alphabets often look alike the ones from the Latin alphabet (= they are homographs). A malicious actor can use this to deceive a victim, for example, to visit a malicious website that pretends to be the legitimate one, a classic phishing attack. A regular user of Facebook may be tricked into to clicking a link where the Latin character “a” is replaced with the Cyrillic character “а”.

You can read more about IDN homograph attacks on Wikipedia.

Maltego Code for A Custom Transform Server 🔗︎

I’ll be demoing the use of my new Maltego golang package here and show how to create a simple Transform server for any tool in very little time.

At the core, all we need is an HTTP service that replies to the request from the Maltego Transform service.

The Maltego package provides a primitive to register http handlers:

func RegisterTransform(handlerFunc http.HandlerFunc, name string)

Since the ditto tool invocations can take longer depending on configuration and your hardware, I configured the http service to never time out. Querying all registered and live imposter domains for common names like amazon etc, without varying TLDs, took between 2-3 minutes for me.

To handle requests to the root page, I used the

func Home(w http.ResponseWriter, r *http.Request)

handler, which will give a simple text based overview of the registered handlers and their routes once visited.

func main() {

	flag.Parse()

	const (
		// ditto status names
		registered = "registered"
		available = "available"
	)

	// register transforms to http.DefaultServeMux

	// all similar domains
	maltego.RegisterTransform(ditto("", false), "similarDomains")

	// only registered domains
	maltego.RegisterTransform(ditto(registered, false), "registeredDomains")

	// only registered domains that resolve to an IP
	maltego.RegisterTransform(ditto(registered, true), "liveDomains")

	// only show domains that are available and not registered
	maltego.RegisterTransform(ditto(available, false), "availableDomains")

	// only live domains that resolve for all TLDs
	maltego.RegisterTransform(ditto(registered, true, "-tld"), "liveDomainsTLD")

	// register catch all handler to serve home page
	http.HandleFunc("/", maltego.Home)

	fmt.Println("serving at", *flagAddr)

	s := &http.Server{
		Addr:              *flagAddr,
		Handler:           http.DefaultServeMux,
		ReadTimeout:       0,
		ReadHeaderTimeout: 0,
		WriteTimeout:      0,
		IdleTimeout:       0,
		MaxHeaderBytes:    0,
	}

	// start server
	err := s.ListenAndServe()
	if err != nil {
		log.Fatal("failed to serve HTTP: ", err)
	}
}

Let’s look at the implementation of ditto invocations:

var ditto = func(status string, hasIP bool, args ...string) http.HandlerFunc {

	return maltego.MakeHandler(func(w http.ResponseWriter, r *http.Request, t *maltego.Transform) {

		// get host that was queried
		host := t.RequestMessage.Entities.Items[0].Value

		fmt.Println("got request from", r.RemoteAddr, "to lookup:", host)

		if !govalidator.IsDNSName(host) {
			t.AddUIMessage("invalid domain: "+host, maltego.UIMessageFatal)
			return
		}

		id, err := cryptoutils.RandomString(10)
		if err != nil {
			fmt.Println("failed to generate id:", err)
			http.Error(w, err.Error(), http.StatusBadRequest)
			return
		}

		file := filepath.Join(storage, id+"-"+host)

		finalArgs := []string{"-domain", host, "-no-progress-bar", "-csv", file, "-throttle=1000", "-workers=4", "-whois"}
		for _, a := range args {
			finalArgs = append(finalArgs, a)
		}

		start := time.Now()

		// run ditto
		// we are running inside a docker container, the ditto binary has been copied into it at build time. 
		// TODO: drop privileges
		out, err := exec.Command("/root/ditto",  finalArgs...).CombinedOutput()
		if err != nil {
			fmt.Println(string(out))
			fmt.Println("failed to run ditto:", err)
			http.Error(w, err.Error(), http.StatusBadRequest)
			return
		}

		data, err := ioutil.ReadFile(file)
		if err != nil {
			fmt.Println("failed to read file:", err)
			http.Error(w, err.Error(), http.StatusBadRequest)
			return
		}

		rd := csv.NewReader(bytes.NewReader(data))
		records, err := rd.ReadAll()
		if err != nil {
			fmt.Println("failed to read CSV records:", err)
			http.Error(w, err.Error(), http.StatusBadRequest)
			return
		}

		fmt.Println("results for", host, "=", len(records), "in", time.Since(start))

		// process results
		for i, rec := range records {
			//fmt.Println(rec)
			if i == 0 {
				// skip CSV header
				continue
			}

			// handle status if provided
			if status != "" {
				if rec[2] != status {
					continue
				}
			}

			if hasIP {
				if rec[3] == "" {
					continue
				}
			}

			addEntity(t, rec)
		}

		_ = os.Remove(file)
	})
}

As you can see, the ditto function is a wrapper to invoke the ditto commandline tool with different flag combinations, by wrapping the

func MakeHandler(handler func(w http.ResponseWriter, r *http.Request, t *Transform)) http.HandlerFunc

primitive to construct transform handlers.

Ditto generates a CSV file as output, which is read, processed, and afterwards deleted again. Some simple filtering is applied based on the desired status or the existence of an IP address before collecting the entities.

Adding entities is done using a simple helper function that takes the CSV output from ditto and assigns the fields to the correct properties:

func addEntity(t *maltego.Transform, rec []string) {
	e := t.AddEntity("dittotrx.IDNDomain", rec[0])
	e.AddProp("unicode", rec[0])
	e.AddProp("ascii", rec[1])
	e.AddProp("status", rec[2])
	e.AddProp("ips", rec[3])
	e.AddProp("names", rec[4])

	e.AddProp("registrar", rec[5])
	e.AddProp("created_at", rec[6])
	e.AddProp("updated_at", rec[7])
	e.AddProp("expires_at", rec[8])
	e.AddProp("nameservers", rec[9])
}

Now that we have some logic for the server side, we need to create a client side Maltego configuration.

Running Maltego Transforms will produce output Entities on a Maltego graph, which are modeled based on the data that ditto provides. In order to further drill down on their data, we will also add a couple of local Transforms. These can be found in cmd/transform:

$ tree cmd/transform/
├── lookupAddr
│   └── main.go
├── toDateCreatedAt
│   └── main.go
├── toDomains
│   └── main.go
├── toIPAddresses
│   └── main.go
├── toNameServers
│   └── main.go
├── toRegistrars
│   └── main.go
└── visitDomain
    └── main.go

7 directories, 7 files

Let’s look at the implementation of a simple local Transform, as a standalone binary:

// This is an example for a local Transform that does a reverse name lookup for a given address.
// It will take an IP address and return the hostnames associated with it as Maltego Entities.
func main() {

	log.Println(os.Args[1:])

	// parse arguments
	lt := maltego.ParseLocalArguments(os.Args[1:])

	// ensure the provided address is valid
	ip := net.ParseIP(lt.Value)
	if ip == nil {
		maltego.Die("invalid ip", lt.Value+" is not a valid IP address")
	}

	// lookup provided ip address
	names, err := net.LookupAddr(lt.Value)
	if err != nil {
		maltego.Die(err.Error(), "failed to lookup address")
	}

	// create new Transform
	t := maltego.Transform{}

	// iterate over lookup results
	for _, host := range names {
		e := t.AddEntity(maltego.DNSName, host)
		e.AddProperty("hostname", "Hostname", maltego.Strict, host)
	}

	t.AddUIMessage("complete", maltego.UIMessageInform)

	// return output to stdout and exit cleanly (exit code 0)
	fmt.Println(t.ReturnOutput())
}

Parsing the commandline args provided by Maltego is done with maltego.ParseLocalArguments. I discard the first element in the args array via slicing because it is the name of the program.

We can then access the main value the Transform shall operate on from lt.Value and try to parse it as an IP. If that fails, the Transform can’t proceed and execution is stopped via maltego.Die, which will exit the program and display an error message to the user in Maltego. If everything is fine, we perform the lookup, construct a new maltego.Transform, add the resulting names as maltego.DNSName Entities (there are constants for the core Maltego types provided in my Maltego go package), and exit the program cleanly using t.ReturnOutput(). That’s it—Less than 20 lines of code and you have a local Transform!

Let’s compile the local Transforms and move their binaries to /usr/local/bin:

go build -o /usr/local/bin/lookupAddr cmd/transform/lookupAddr/*.go
go build -o /usr/local/bin/toDateCreatedAt cmd/transform/toDateCreatedAt/*.go
go build -o /usr/local/bin/toDomains cmd/transform/toDomains/*.go
go build -o /usr/local/bin/toIPAddresses cmd/transform/toIPAddresses/*.go
go build -o /usr/local/bin/toNameServers cmd/transform/toNameServers/*.go
go build -o /usr/local/bin/toRegistrars cmd/transform/toRegistrars/*.go
go build -o /usr/local/bin/visitDomain cmd/transform/visitDomain/*.go

I’ve created a small helper tool to generate Maltego Entities and Transforms based on a YAML configuration since configuring everything via the UI can be tedious. I’ll cover the details of this tool in another blog post. For now, we will just look at a simple example.

The following yaml is used to generate the Maltego configuration:

org: DittoTRX
author: Philipp Mieden
description: Transformations for the ditto tool

entities:
  - name: IDNDomain
    image:
      name: domain
      color: black
    description: A domain likely used for phishing
    parent: maltego.Domain
    fields:
      - name: unicode
        description: Unicode representation of domain name
      - name: ascii
        description: ASCII representation of domain name
      - name: status
        description: Registration status of domain name
      - name: ips
        description: IPs for the domain name
      - name: names
        description: DNS names that resolve to this domains IPs
      - name: registrar
        description: The name of the registrar
      - name: created_at
        description: The date of creation
      - name: updated_at
        description: The last update
      - name: expires_at
        description: The date of expiry
      - name: nameservers
        description: Nameservers associated with domain

workingDir: ~

transforms:
  - id: LookupAddr
    input: maltego.IPv4Address
    description: Lookup Address
    executable: /usr/local/bin/lookupAddr
  - id: ToDomainNames
    input: dittotrx.IDNDomain
    description: To Domain Names
    executable: /usr/local/bin/toDomains
  - id: ToRegistrarNames
    input: dittotrx.IDNDomain
    description: To Registrar Names
    executable: /usr/local/bin/toRegistrars
  - id: ToNameServers
    input: dittotrx.IDNDomain
    description: To Name Servers
    executable: /usr/local/bin/toNameServers
  - id: ToCreationDate
    input: dittotrx.IDNDomain
    description: To Creation Date
    executable: /usr/local/bin/toDateCreatedAt
  - id: ToIPAddresses
    input: dittotrx.IDNDomain
    description: To IP Addresses
    executable: /usr/local/bin/toIPAddresses
  - id: VisitDomain
    input: dittotrx.IDNDomain
    description: Open Domain in default Browser
    executable: /usr/local/bin/visitDomain

As you can see, the Transforms all operate on our custom entity type IDNDomain and have a description and an executable set.

To generate the config, all we need to run is:

$ maltego-gen maltego.yml
material icon repository exists, pulling
bootstrapped configuration archive for Maltego
packing maltego dittotrx archive
packed maltego dittotrx archive
copied generated file to /Users/you/DittoTRX.mtz

The resulting .mtz archive can now be imported with Maltego:

Now all that is left is to head to your Transform Distribution Server and add the Transforms. For simplicity, I am using a basic authentication, which is protected over TLS. Future versions could use oauth as suggested by Maltego.

You can read more about how to set up a TDS or add Transforms in the excellent Maltego docs or Maltego’s Complete Guide for Building Custom Maltego Integrations.

Using Ditto Transforms in Maltego 🔗︎

In Part 2 of this article series, I will demonstrate how you can use these ditto Transforms in Matlego for domain investigations.

READ PART 2

Follow Maltego on Twitter and LinkedIn and subscribe to Maltego’s email newsletter to stay updated on the latest releases, tutorials, and webinars!

About the Author 🔗︎

Philipp Mieden

Philipp Mieden is a security researcher and software engineer from Germany, currently focusing on network security monitoring and the use of machine learning. He presented his research on classifying malicious behavior in network traffic at several international contests from Kaspersky Lab and won multiple prizes. After finishing his bachelor at the LMU Munich, he moved to Amsterdam to continue his master studies in “Security and Network Engineering” at the UvA. He is sharing many of his projects on Github, but you can also find him on ResearchGate or Twitter. Besides network monitoring, Philipp is also interested in hardware security, industrial control systems and reverse engineering.

Learn more about Philipp’s work on his Twitter or GitHub.