Communicating Remotely

(Thanks to Nikki Heald and Dave Murphy for their contributions)

For about 7 years, I worked from home participating in a very large Open Source project  (Ubuntu Linux). Since then, I have continued setting up and advising companies on how to work remotely.

While you might be struggling with setting up on short notice your working environment at home, soon you will find your groove, and then the next big thing will hit you: Your usual ways of accessing information and the tools you use to communicate have suddenly, drastically changed.

When you were working in an office, you used Instant Messaging (IM) to communicate with people in other offices every now and then, but you largely relied on face to face conversations or office meetings. Now EVERYTHING seems to happen in IM.

Instant Messaging (IM) is a great tool for remote working but also has massive pitfalls. This is not just your problem but something remote organisations have been struggling with for years. Here are some key best practices:

Keep communication in the open. Set up team chats and invite people from other teams to ask questions in public channels, avoid as much as possible starting conversations on direct and private messages.

It is extremely easy to misinterpret and get frustrated using instant messaging. Keeping conversations in the open makes people think twice about the language they use. It also increases the possibility of someone from the group chat replying to your urgent message faster.

Be kind and patient.  Written communications are always easier for non-native speakers, but it can also make us overestimate the language skills of our colleagues and read too much into what they write. It might be that something you feel is aggressive or even offensive is just a side effect of this. At this point, take a breath, be polite and ask to have a video call.

Move detailed conversations to private messages and then to video call.  The recommended communication flow is to start in an open forum (group chat) , then if the conversation gets into details and it is clogging the channel, ask for it to be moved to a thread or private direct message, finally if still taking too long, set up a video call to address the issue at hand.

IM is asynchronous, really!. Instant messaging gives us the false sense that we have the full attention of our colleague, and that they are going to reply straight away. This is nothing further from the truth! Work or personal life can be interrupting and stressing out the person you are talking to, leading to long pauses or terse replies. Do not get frustrated, relax, they will get back to you as soon as they can.

IM is disruptive. Constant direct IM interruptions can lead to low productivity, this is another reason to use group chats. One good tip for teams is to designate a person (maybe rotate it daily) responsible for answering questions from other teams in your group chat. This means only one of you is getting distracted, but also gives a better responsiveness to the person asking the question.

Document decision outside IM. Instant messaging is ephemeral, even if it is in written form. If you take a decision in a group or private chat, follow it up with an email, a gdoc or whatever else your team uses to share documentation.

Don’t try to read everything. Very quickly IM proliferates and becomes an avalanche of information. Don’t have fear of missing out, you shouldn’t expect to read everyone’s messages AND don’t expect everyone to read your messages. If there is something that you really need people to read, use an email or arrange a meeting.

IM is not that social. Although it is a great tool for GIFs and Memes, it is not going to replace the social interaction that you are used to in the office. Instead, try leaving a video call, such as hangouts, open with the rest of your team members while you work on your tasks, just for chit-chat.

If you have any other questions or you are struggling with managing your team communications while working remotely, don’t hesitate to reach out to me!

 

Developing Yourself at Ebury

One of the key things that motivate me every day to work at Ebury is the company’s passion to invest in its people. This comes directly from Juan and Salva (our founders) and it is lived by all of us in management positions.

But what does it mean for our Tech teams? Originally, when we were 10-20 people in Tech, this happened organically, but over time and with growth, it is important that career development also scales up and it is consistent across teams. 

During the last year and a half, I have been working with the team to define a framework that allows people to plan their development and have meaningful conversations with their leads.

At Ebury Tech, your career can take several paths, but it all starts with mapping what are 6 competencies that we really care about.

chart

6 Competencies and 4 Levels

We have summarised our cultural values into 6 competencies, each of them with 4 levels. Leve 1,2 tent to focus on you, as an individual, while the others focus on your impact in your team(3) and the wider tech organisation(4):

  • Domain Mastery: How good are you in your domain and do others agree? 
  • Team Work: Do you help to achieve others’ objectives? 
  • Continuously Improves: Do you improve and help others improve?
  • Problem Solving: Can you resolve real-life issues with a simple solution?
  • Business Impact: Deliver value, not just code
  • Leadership: Do you set an example to others and take responsibility?

Not everyone  follows the same path

Although it might sound like something that you get from a fortune cookie, it really applies to personal development. At Ebury, we have define 3 loose paths to guide your development in Technology:

People Leadership – This is the path I chose, while still technical, my passion is to help others develop and work with team to achieve high performance

Technical Leadership (Architecture)Another available path is to focus on your ability to design systems that deliver customer value and to work with others across the organisation to hone the best solutions for our clients.

Technical Leadership (Hands on)Finally, there is also a path to lead by example, to bring change and best practices from within a squad. 

Screenshot 2020-01-23 at 14.38.13

Putting it all together

Each step in a development path is defined by a set of behavioral and observable expectations. These are a level within each of the 6 competencies.

Note, that I have said observable. This is important because, in order to move from one step to another, your team lead will put you forward a case with examples to a panel of peers that will support your change based on observable facts.  Not everyone will move to the next step at the first try, but you will receive tangible feedback on how to achieve it next time. 

Screenshot 2020-01-23 at 14.51.35

Digg deep, let me know what you think

There is no secret sauce to what we do to invest in our people, hence I am sharing with you and I look forward to your feedback. Hit me up in Linkedin

Ebury: The Fin and the Tech

We often talk about Ebury being a disruptive Fintech company. Today I wanted to go into more detail about what makes Ebury a Financial and Technology company.

FINtech

Ebury’s core business is foreign currency exchange (Forex), and we focus on the enterprise segment of the market. To better illustrate what we do, let’s take as an example a made-up European toy company, which we will call TOYSA.

TOYSA’s biggest season is the Christmas holiday period between December and January. TOYSA sells toys to many countries in Europe including the UK. Their UK sales channel received large orders for their new drone, the TOYSA D500. Their UK office would like to know what should be the retail price in British Pounds (GBP). The TOYSA D500 is produced in a factory in Shenzen, China and a single unit to make costs 790 Chinese Yuan. Let’s say that “today” that is equivalent to 100 Euros. TOYSA would like a 20% margin on their toys, so they retail at 120 Euros in mainland Europe, that “today” translates to 110 GBP.

In order to have the toys ready for sale in December, they need to be ordered in June so they can be manufactured in August, and TOYSA is going to be running a big marketing TV advertising campaign in October.

But TOYSA’s CFO can’t sleep at night… she is worried. What if the Yuan grows strong against the Euro from June to August? Their 100 Euros cost calculation could be completely wrong!! Conversely, with Brexit looming, the British Pound could devalue against the Euro, a retail price of 110GBP could not even cover the cost of manufacturing…

This is where Ebury can help TOYSA. We offer both Spot (right now) and Forward (any time up to two years) contracts that remove the risk of currency fluctuations by providing a fixed exchange rate for a future date. TOYSA’s CFO can use these rates to calculate her profit and retail prices removing risk from the equation.

But… hold on! She is still worrying. She needs to pay for the shipment of drones with money that will come from future D500 sales, which she doesn’t yet have! Yup… Ebury can also help with that. It is called Trade Finance.

finTECH

Ok, now you understand what we do but how do we do it with over 140 currencies? With quite a bit of Django/Python.

Our back-end services, which all run on Amazon EC2 (some of them as part of ECS clusters), take care of financial tasks such as getting quotes for currency exchanges, instructing payments, receiving funds, Anti-Money Laundry checks, reconciliation of accounts, and so on. All new services developed in the last few years run in AWS Elastic Container Service (ECS) and we are right now, in the process of containerising older services.

Customers contract Ebury’s products in 3 different ways:

  • They contact their Ebury account manager. Our Front-office team is pretty large, and to ensure consistency, we automate our sales and operations processes in Salesforce. Some of our back-end systems also have “admin” level consoles for financial reports and complex operations, these are built as Django applications.
  • They use Ebury Online (EBO). online.ebury.com is a Javascript application served by a Django app, that provides a two-factor authentication login into a front-end for customers to manage their transactions and book new currency trades. It connects to our back-end services internal API.
  • They use their own ERP (such as Netsuite). Netsuite is just one of the ways customers can access our REST API, which is a Flask service that provides a programmatic interface to the same internal back-end API used by EBO.

Join Us!

If all of this sounds interesting and you are looking for a new opportunity, we are hiring!! Whether it’s a remote position you are after or you would like to join us in our Central Malaga office, get in touch! We could have the role for you.

Story Estimation

We were having a chat internally at Bitnami with respect: should we try to standardise T-Shirt sizes across the team and also if we ought to have a mapping between user story points and implementation hours.

I felt that this is a discussion that keeps repeating, in conversations internally and externally, and I thought it would be worth it to put some effort into summarising my  thoughts about it.

Story Points vs Task Hours

Normally you will find Scrum coaches recommending the use of T-Shirt or Points for Story sizing, and hours for estimating Tasks. Stories are the unit of product/user value that it is delivered (tasks are just work).

Story sizing (T-Shirt or Points) is not an estimation of time, rather a way to express complexity. Yes, there is a correlation between complexity, effort and hours.  However, Complexity tends to be less variable across team members than hours.

A Story is complex due to the problems to be solved while implementing it, but the time it takes to solve the same problem might vary depending on the skills and experience of the person solving it.

OK, but how do we established consistency in estimating complexity? Human beings are pretty good at comparing similar problems. Problem A is similar to Problem B. So, I normally start by getting a team to agree the sizes of stories/problems that they have completed/solved together in the past. Then use that as reference point going forward.  This common understanding can only be shared by people in the same squad, since it is required a shared experience.

Story sizing can be and it is used to measure velocity, and also burn down progress. However, IMHO they are most useful to identify misalignment (Foo thinks is XXL but Bar thinks is L, how come? maybe Bar has some context that is missing Foo? or the other way around?). This conversation between team members about complexity is where the real value of Story sizing resides.

I think we can all agree that we can not predict the future.  Hence estimating in hours is always highly inaccurate. Inaccuracy grows as you extend the period of time you are estimating over (i.e. something that can take months or it is months away).

So why is it OK to estimate tasks in hours? Because tasks are small break down of stories that are assigned to an individual to implement.  Basically, the estimation is done over the shortest time span possible and taking the skills and experience of that person into account.

Tracking and managing cost

A different (and valuable) problem to solve is tracking the cost of implementing a feature/product/release for financial reasons. Most of us work on either For Profit or Cost Driven (non-for-profit) organisations, so having a record and an indication of where money (and hence hours) are going can be critical for our organisations.

While estimating time is highly inaccurate,  tracking actual time spent on tasks doesn’t have to be. Clearly there is a (human) cost associated on the granularity of this tracking. Which is the right level of tracking will be highly dependent on your business and accounting model.

Enhancing Helm Charts with Operators

I was interested to see if I could blend a Helm Chart (packing and deployment) and an Operator (day-2 operations) in a single solution, so I developed a proof of concept for ChartMuseum.

I have been wanting to play with the new Operator SDK from RedHat for a while. I like the concept of having an api for the life-cycle management. IMHO, operators are a development pattern that matches declarative Kubernetes API objects to lifecycle events like backup, restore, and other type of maintenance tasks.

In general, most operator implementations that I have played with also take care of the deployment of application. In some cases, like the etcd operator, they manage the life-cycle of pods directly rather than using standard objects like deployments or replicasets .

I have been doing some Helm Chart development and I really like the flexibility that Helm gives you to parametrise deployments.  It seemed to me that I could still use a Helm Chart for packing and deployment, and enhance it with an operator to be included in the releases to manage the application management beyond deployment.

As a proof of concept, I decided to try to extend the existing upstream chartmuseum chart with an operator that took care of adding and packaging charts from a github repo.

The basic operations that I set up to automate were:

  • Pulling a new git repository and helm packaging its contents
  • Regularly pulling updates and repackage a repo
  • Removing a git repository and its contents

I needed to be able to ask the chartmuseum to perform these activities. Using the adapter/sidecar pattern, I developed a new container to expose these as http endpoints and package dependencies (git and helm) to be bolted to chartmuseum’s container within the same pod.

So a Custom Resource for this operator will look like this:

apiVersion: "cm.bitnami.com/v1alpha1"
kind: "Chartmuseum"
metadata:
  name: "myrepo"
spec:
  git: "https://github.com/foo/mycharts"
  updateEveryMinutes: 15
  dependencies:
  - name: bitnami
    url: "https://charts.bitnami.com/bitnami"

The git repository has to be publicly available. The helm charts might be pulling dependencies from other repos, so I also added the ability to define these.

updateEveryMinutes is an integer value that indicates how often, in minutes, the git repo should be updated. Instead of having this functionality baked into the operator, it creates Cronjob objects to trigger the update. This objects are gettable and visible to the user.

On creation of a new custom resource of type Chartmuseum, the operator will:

  • Add dependencies as required –  POST /repo/dependency
  • Then add a new repository, which will trigger a git clone, and a packaging of all folders in the repo that contain a “Chart.yaml” file –  POST /repo/new
  • A new Cronjob will be created with the same name and namespace than the Custom Resource that will updateEveryMinutes hit GET /repo/name/update

On deletion of the custom resource, the operator will:

  • Delete the repo and its artifacts  – DELETE /repo/name
  • Delete the Cronjob object

Incorporating this into an existing Helm Chart was relatively simple. I created a new value flag called operator.enabled, which if set to true will add additional manifests to the release by:

  • modifying the main deployment to add the sidecar container,
  • modifying the main service to expose a new port for sync operations,
  • adding CRD for Chartmuseum and a deployment for the Operator,
  • adding RBAC support for the operator to read CRD and create Cronjobs

In summary, I found the operator-sdk really simple to use and work with. It really takes away the fiddly parts of creating an operator and lets you focus on the logic. I also found that the operator and the helm chart worked nicely together and provided a good transparency and control to the user.

If you want to try it, just clone this repo: https://github.com/vtuson/chartmuseum-chart and run:

helm install . -n mychartmuseum

Scaling WordPress In Kubernetes

Cloud Native applications have been designed to be run in microservices architecture where individual components can be scaled separately, data is persisted and sync across replicas and node failures can be easily survived. This can be more tricky with traditional web applications that have not been designed this way, such as WordPress.

Bitnami produces production-ready Docker images and Kubernetes Helm Charts that you can deploy into your Kubernetes Cluster. This blog will make reference to concepts that have been previously covered by our Kubernetes Get Started Guide.

WordPress Is A Stateful App

Although WordPress uses a database backend to persist and maintain user created content, administration changes are persisted to the local drive. These includes the plugins installed, site configuration and CSS/Media used by the web front-end. Multiple replicas of the same WordPress site would expect to have access to a shared drive. Basically, WordPress likes to keep its state close by.

Our Kubernetes Helm Chart for WordPress uses a Deployment to manage and scale the WordPress pods. In order to share this admin-created content across pods, the deployment mounts a volume provisioned by a Persistent Volume Claim(PVC).

When scaling a web app it is important to understand the different types of Access Modes available for persistent storage in Kubernetes: ReadWriteOnce (RWO), ReadOnlyMany (ROX) and ReadWriteMany (RWX).

If you want to administer your WordPress site separately, you could expose a read-only configuration volume to your pods. In this case, ROX would be you best choice since it can be mounted as read-only by many pods across multiple nodes.

If you want to continue using the WordPress admin interface in all its glory, all your pods will need read-write access to a common volume. It is also likely that you would like your pods run in different nodes (for better site availability and scalability). Since a RWO volume can only be mounted in one node at the time, you would really need to use RWX.

Great, Where Do I Get RWX Volume?!

Unfortunately RWX is not a very commonly supported access mode by the current list of volume plugins (see official documentation). So what can you do if you don’t have access to one in your cluster?

Since, WordPress write access does not require a highly performant solution, we are going to share a RWO volume across multiple pods by remapping it as a RWX NFS volume.

Hold on, this is going to be complicated, right? Nope, it is going to be a single command.

There Is A New Chart In Town

A few days back, a new Helm Chart landed on the official Kubernetes repository – Introducing the NFS Server Provisioner.

What this chart does is to deploy all the bits you need to enable dynamically serving NFS persistent volumes (PV) from any other volume type. Most Kubernetes deployment will have a Storage Class that provides RWO volumes. We are going to map a single RWO persistent volume from this existing Storage Class and share it as RWX Storage Class called ‘nfs’.

The following diagram show a summary of the kubernetes objects involved in this process:diagram

Please note that this solution has low fault tolerance, as an outage of the node that has the RWO mounted will affect the whole deployment availability (should not lose data).

As promised, deploying the NFS-Server provisioner is a simple helm command:

$ helm install stable/nfs-server-provisioner --set persistence.enabled=true,persistence.size=10Gi

Deploying WordPress To Work With NFS

Once you have deployed the Helm release, you can check that a new Storage Class called ‘nfs’ is now available in your cluster:

$ kubectl get storageclass

show storageclass results

Also the RXO volume claim has been created. In this case, it is using the standard storage class in minikube (hostpath) and it is bound to the NFS-server statefulset.show rxo pvc

Next we can deploy a WordPress release, using the NFS Storage Class:

$ helm install  --set persistence.accessMode=ReadWriteMany,persistence.storageClass=nfs stable/wordpress  -n scalable

We can see how we now have 3 PVCs in our cluster:

  • a RWO for the MariaDB backend database
  • a RWO for the NFS Server
  • A RWX for the WordPress pods

Inspecting the new PV, we can see that it is served over NFS:nfs pv

Are You Ready To Scale Horizontally?

Almost, the WordPress admin console will require a user session to be served always by the same pod. If you are using an ingress resource (also configurable by the chart) with a Nginx ingress controller, you can define session affinity via annotations. Just add these annotations to your values.yaml file, under ingress.hosts[0].annotations.

Now you just need to scale your WordPress deployment:

$ kubectl scale deploy scalable-wordpress --replicas=3

Happy Highly Available Blogging!

Introducting TUI – a library for building simple text interfaces

Why Tui?

At Bitnami, we have a bunch of useful command line tools that can be used to operate our stacks. I wanted to build a basic menu for the most used, so I didn’t have to remember each command and each argument. Then I thought It would be great to have a library that would work a bit like the cli go libraries, you can define the inputs and the function to handle the command and the user experience ‘just happens’. So that is why I created TUI

What is Tui?

This is a Basic Implementation of a text ui D.O.S style, that can be used to run CLI commands or you can define your own HandlerCommand to run Go functions. You can find it in my Github repo: https://github.com/vtuson/tui

Basic Structure is:

  • Menu
  • Commands – Each menu can have a set of commands to present
  • Args – Each command can have a set of arguments that get passed

The idea behing TUI is provide a simple library similar to the cli libs like https://github.com/urfave/cli that allows you to build a menu based application for terminal users.

By default a CLI command handler is provided that is able to pass parameters as flag, options or Enviroment variables You can find an example app in the /sampleapp folder

screenshot

Sample app

The project comes with a sample app under the folder /sampleapp

 cd sampleapp
 godep get
 go run app.go

Getting started with tui

A basic shell

Lets write a small application, You need to import the package, and create a basic menu with no option:

package main

import (
   "github.com/vtuson/tui"
)

func main() {
   menu := tui.NewMenu(tui.DefaultStyle())
   menu.Title = "Test"
   menu.Description = "Test app"
   menu.Show()

// This function handles input events like key strokes
   go menu.EventManager()

//Menu contains a channel that lets you know when the user has exited it
   <-menu.Wait

//Quits the menu, and cleans the screen
   menu.Quit()
}

Adding a command without arguments

Now lets add a small OS command that will run without need for arguments.

package main

import (
	"github.com/vtuson/tui"
)

func main() {
	menu := tui.NewMenu(tui.DefaultStyle())
	menu.Title = "Test"
	menu.Description = "Test app"

	menu.Commands = []tui.Command{
		tui.Command{
			Title:       "No Args",
			Cli:         "echo hello!",
			Description: "just being polite",
			Success:     "Yey it works",
			PrintOut:    true,
		},
	}

	menu.Show()
	go menu.EventManager()
	<-menu.Wait
	menu.Quit()
}

The option Printout allows for the output of the command to be shown to the user. The Description and Success are strings that will be use to add more context to the command execution. They are optional and dont need to be set if you don’t want to.

Commands with arguments

You can also add argument to command that a user can input

package main

import (
	"github.com/vtuson/tui"
)

func main() {
	menu := tui.NewMenu(tui.DefaultStyle())
	menu.Title = "Test"
	menu.Description = "Test app"

	menu.Commands = []tui.Command{
		tui.Command{
			Title:       "Args CLI",
			Cli:         "echo",
			PrintOut:    true,
			Description: "test of running a tui.Command with arguments",
			Args: []tui.Argument{
				tui.Argument{
					Title: "Please say hi",
				},
			},
		},
	}

	menu.Show()
	go menu.EventManager()
	<-menu.Wait
	menu.Quit()
}

Arguments can also be boolean flags, they can have a name (which can be passed as input to the command) or can be set as environment variables.

Adding your own handler

The library assumes that the command is an OS command to be executed using the provided OSCmdHandler function if the Execute is nil. If another handler is passed by setting the Excute value, then it will be called when the users selects that command.

Here is a simple echo example:

package main

import (
	"github.com/vtuson/tui"
	"errors"
)

//This is the customer handler c is the command object 
//for the command to be executed. 
//ch is the channel that can be used to return text to be displayed back to the user.
//The handler is executed async from the main program, 
//so you must close the channel when completed or the menu will hang 
//waiting for your command to complete.

func CustomHandler(c *tui.Command, ch chan string) {
	//defer close to make sure when you exit the menu knows to continue giving user the output
	defer close(ch)
	for _, a := range c.Args {
		ch <- a.Value
	}
	//if you want the command to fail set Error in the command variable
	c.Error=errors.New("It failed!")
}

func main() {
	menu := tui.NewMenu(tui.DefaultStyle())
	menu.Title = "Test"
	menu.Description = "Test app"

	menu.Commands = []tui.Command{
		tui.Command{
			Execute:     CustomHandler,
			Title:       "Args CLI",
			Cli:         "echo",
			PrintOut:    true,
			Description: "test of running a tui.Command with arguments",
			Args: []tui.Argument{
				tui.Argument{
					Title: "Please say hi",
				},
			},
		},
	}

	menu.Show()
	go menu.EventManager()
	<-menu.Wait
	menu.Quit()
}

I took a circular saw to the Nextcloud box and you won’t believe what happened next!

Ok, ok.. sorry for the click-bait headline – but It is mainly true.. I recently got a Nextcloud box , it was pretty easy to set up and here are some great instructions.

But this box is not just a Nextcloud box, it is  a box of unlimited possibilities. In just a few hours I added to my personal cloud  a WIFI access point and  chat server.   So here are some amazing facts you should know about Ubuntu and snaps:

Amazing fact #1 – One box, many apps

With snaps you can transform you single function device, into a box of tricks. You can add software to extend its functionality after you have made it. In this case I created an WIFI access point and added a Rocketchat server to it.

You can release a drone without autonomous capabilities, and once you are sure that you have nailed, you can publish a new app for it… or even sale a pro-version autopilot snap.

You can add an inexpensive Zigbee and Bluetooth module to your home router, and partner with a security firm to provide home surveillance services.. The possibilities are endless.

Amazing fact #2 – Many boxes, One heart

Maybe an infinite box of tricks is attractive to a geek like me,  but what it is interesting is product makers is :make one hardware, ship many products.

Compute parts (cpu,memory,storage) make a large part of  bill of materials of any smart device. So does validation and integration of this components with your software base… and then you need to provide updates for the OS and the kernel for years to come.

What if I told you could build (or buy) a single multi-function core – pre-integrated with a Linux OS  and use it to make drones, home routers, digital advertisement signs, industrial and home automation hubs, base stations, DSLAMs, top-of-rack switches,…

This is the real power of Ubuntu Core, with the OS and kernel being their own snaps – you can be sure the nothing has changes in them across these devices, and that you can reliably update of them.  You not only are able to share validation and maintenance cost across multiple projects, you would be able to increase the volume of your part order and get a better price.

20160927_101912

How was the box of tricks made:

Ingredients for the WIFI ap:

 

I also installed the Rocketchat server  snap for the store.