Join Our New Slack Community For Gophers

By William Kennedy

12 May 2014

Why Are We Doing This?

GopherCon was an amazing experience for many of us who had the privilege to go. We met so many great people. Some for the first time and many who we had relationships with on social sites such as Twitter and G+. On my last night in Denver I started to feel sad. I didn’t want to lose that camaraderie I had made with these friends. I just kept thinking, how could this be sustained?

After a couple of weeks of experimenting and talking with friends, Ed Gonzalez introduced me to a product called Slack. Within 5 minutes we registered a group, invited our friends and asked a few of them to be admins. Within a couple of days we had 25 people in the community. Everyone was really excited about the platform and the possibilities it brought to community building.

We thought it would be awesome to publicly open up the group to the community. Slack is an invite only system, so to make the community inclusive we recognized the need for a central authority that everyone would trust. We also wanted to put a structure in place that would minimize and monitor those who wanted to be a disruption. After many discussions we came up with following structure:

Gopher Academy (GA) will be the central authority for the group.

GA has the trust of the community and is a place for those who want to teach, build community or need help with events. GA currently manages a successful blog, the GopherCon conference, and recently the Go Newsletter has joined their efforts.

Members will be assigned as administrators to help monitor conduct and add new users.

GA is too small to do this alone. When a need arises, GA will recruit people to be administrators. Choosing administrators will be based on the number of members and the different time zones and activity. The only function of adminstrators is to get new members online and monitor compliance with the code of conduct.

Following people have been assigned as administrators.

Brian Ketelsen, Erik St. Martin, Cory LaNou, Levi Cook, Chris Schaefer, Nathan Youngman, Trevor Bramble, Julia Poladsky, Ed Gonzalez, Satish Talim and William Kennedy.

Members are required to follow the code of conduct.

This code of conduct has been published and all members are required to be good citizens. Those who violate the rules will be subject to being revoked from the community.

You Have Questions?

We hope many of you consider joining the community. We think Slack brings something special that you will recognize as soon as you join. We know that you will have questions and we want to do our best to answer them. Here are a few questions that we had:

Why are we using Slack and not IRC?

Slack has native clients for desktop and mobile devices. The features that Slack brings and their tools really enhance community building. We think members will get more out of Slack than IRC. Slack allows easy pasting of code snippets, files, and longer text posts.

How can I be invited to join?

GA has setup an email address, slack@gopheracademy.com. Just send your request to that email address and one of the administrators will pick it up and send you an invitation.

How can I become an administrator?

GA will determine when a need arises to add more administrators. This will be based on the number of members the community has to service and the different time zones and activity. If you are interested in being an administrator, please submit your request to GA.

Does being an admin give that person God-like power?

No, it's just a group of people who will help on-board new members and ensure that the code of conduct is being followed. It's not a popularity contest or a social club.

GopherCon 2014 Retrospective

By Brian Ketelsen

2 May 2014

Denver, Colorado - 2014

Gophercon 2014 is in the books. In the end, we had 700 attendees, and it looks to us like everyone enjoyed the experience. Putting on a conference is really hard work, but once the date arrived everything seemed to come together well. It was really gratifying for us to see so many Go enthusiasts at the event, and I think the best part of the conference was getting to meet all the great people in the Go community that I've been interacting with for the past four years. A conference was certainly overdue.

Credit Where It Is Due

While the bulk of the organizing was done by Erik St. Martin and me, we had some tremendous help from people in the community. I'd like to single some of them out here for extra thanks:

  • Cory LaNou, Levi Cook, Julia Allyce and the rest of the Denver Gophers. They were amazing, providing a place to store hundreds of boxes of Gophers, Mugs, t-shirts, programs, and more, topping it off by helping to prepare the bags, then staffing the registration desk for us during the whole conference. Can't say enough about the awesome Go community in Denver.
  • Steve St. Martin -- Erik's brother came along to help wherever he could and ended up practically running all the registration and pre-conference operations. Thank you!
  • Kelsey Hightower saw that Erik and I were too busy running around lining up the next speakers to pay good attention to the job of keeping the crowd engaged and giving the speakers the warm welcome that they deserved. He volunteered to jump in as emcee, and did a fantastic job of keeping the conference moving on schedule and engaging the audience. He'll likely become the Billy Crystal of Gophercon, with a permanent job if he's willing.
  • Damian Gryski, Nathan Youngman, Andy Walker, Dave Cheney, Chris Schaefer -- Our pre-conference frenzy of bag stuffing would have taken us three weeks if it wasn't for these community members jumping in and lending a hand. Everyone was tired and sore from stuffing 700 conference bags by the end of Wednesday, and most of us missed the pre-party, but we made it happen. Thank you all for spending valuable time getting things ready for the crowds.
  • Erick Zayala from Ardan Studios did all of the artwork for our programs, mugs, pens, shirts and signs. They came out so well!
  • Andrew Gerrand from Google. Andrew believed in our vision early and provided moral support and advice along the way.
  • All of our Sponsors We set out to create an event that would be affordable for anyone, from students to enterprise developers, and we couldn't have done that without the generous sponsorship that these companies provided.
  • Everyone who bought a ticket for the Gophercon Women's Scholarship. We were able to distribute every ticket, and we're proud of the community for fostering an open and inclusive environment. One sponsor bought 5 tickets, and I'm pretty sure he paid with personal funds. Thank you, thank you, thank you.
  • And of course, our amazing speakers. We spent a lot of time picking speakers and topics that would provide a broad range of information to people with various levels of exposure to Go. Our speakers really delivered, providing great advice, demonstrations, technical overviews, background, and best practices. I learned things from every talk.

There are so many other people to thank. Abby Smith and Christine St. Martin were amazing, helping out wherever needed. So many community members helped in large and small ways. To all of you, our heartfelt thanks. It takes a village to raise a Gopher, and we've got a pretty amazing village.

What Went Well

We've already touched on some of the things that went well: Our speakers were awesome, the community was engaged, the volunteers carried the day. We also thought the Marriott did a great job of responding to our needs. The first day, it was a little confusing during lunch for those people who had special dietary needs. On day two, the event concierge helped us coordinate all those meals into one spot so it was easier for the attendees to find their meals. The Marriott staff also pulled a rabbit out of their hat, providing us with an unplanned lunch on Saturday for our "Hack Day". Only 50 people responded that they planned on attending, so when we had almost 500 people show up we were taken off guard. Michael from Marriott catering pulled a last-minute lunch off without a hitch. Thank you to Google and Digital Ocean for volunteering to help pay for that lunch. I think it worked out really well, as everyone spent some time socializing during lunch. It was a nice vibe for the mostly-informal ending day of the conference.

Speaking of the "Hack Day"... that was another hit. We were amazed by the quality and quantity of the lightning talks, and on the other side of the room it was so much fun to watch people working together on projects. The Hack Day was something Erik wanted to do from day one, and now I certainly see the light. It was a great time.

Where We Can Improve

We initially planned on 500 people, but the crushing demand as we sold out prompted us to re-arrange our space planning to find room for more people. It meant that our lunch was served in two separate rooms, and that was a little disjointed. Next year, we'll find a good target number of attendees and plan for that crowd from the beginning. That's a balancing act, since too many people will make the conference impersonal. We're thinking about capping attendance around 1000 or 1200 next year. That may mean that we need to change venues. We haven't begun the planning process for 2015 yet, but we will be doing that in the coming weeks.

We'd also like to see a wider variety of activities outside the core hours of the conference. This year we had pre-parties, pub crawls, coffee shop walks, gaming, hack sessions and more. Next year, we'd love to see more community organized activities that give people an opportunity to get out and make lasting friendships.

We'd like to find a way to let our sponsors engage better with the attendees. Too many conferences have open exhibit halls where the attendees are trapped during breaks. All of our sponsors this year spent a lot of time directly engaging with the attendees, providing tutorials and live demonstrations. We'd like to find a way to make that more comfortable for everyone and facilitate that engagement without turning it into a big trade show.

For a conference that went out of its way to promote and encourage the attendance of women, we somehow didn't manage to have women's sizes in our t-shirts. Mea Culpa. We'll certainly get that right next year.

Where We Are Going From Here

This conference is just the first of hopefully many more. We're busy trying to think of other ways we can help educate and build the community around Go. If you have ideas, or you want to become more involved, drop us an email -- social@gopheracademy.com Our mission is to foster the growth of the Go community.

Thank You All

Thanks to everyone who came! We were humbled by the outpouring of gratitude from everyone who went to Denver, and we can't wait to see you all again next year. Watch this blog for links to the presentation videos when they're ready.

Erik St. Martin

Brian Ketelsen

Gogs: Binary Deployment: The Right Way to Deploy

By Jiahua Chen

1 April 2014

This post is published corresponding to the Gogs - Go Git Service v0.2.0 release.

First, please let me speak for the develop team to thank all of our friends who are supporting us on GitHub. As you may know, v0.2.0 is the first public release of Gogs, and the community has contributed over 650 stars to this project on GitHub in just one week.

As for me, I want to give the most sincere gratitude to all members of develop team, everybody worked extremely hard for the first public release. Our united and tacit understanding is the key to the success of this project.

Project Overview

We want to be clear with this, the first public release, to help you understand more about why we launched this project, how we develop it, and its current develop status.

Purpose

In the area of self-hosted Git services, there are some very succeesful products running all around the world, so why did we decide to develop Gogs? How can we compete with them(e.g. gitlab)? What's the view of us in terms of recreating the wheel?

Officially, our purpose is to take the advantage of Go building everything into one binary to setup your own self hosted Git Service. But in my words, I just don't like the products that exist right now. Because they need a lot of installation steps and dependencies, or they come from a single developer with no anticipated prospect, or they aren't cross-platform. And Gogs certainly supports a rich number of operating systems because of Go's cross-platform reach.

We have at least two competitors in the long term, gitlab and GitHub Enterprise. So what does Gogs have to compete with them? I think right now we have only one advantage: the binary deployment. Setting up your git service simply will be our big competitive advantage.

Many people just hate so-called "creating the wheel repeatedly" without any reason, then they miss the opportunity to be creative. You need to see it in several perspectives. First of all, it's a great thing for learning and practicing skills, especially try something you've never got a chance to try, like we chose martini as our basic framework. Secondly, current services do not make us feel satisfied, and we won't give up the chance to improve it.

Develop Team

Gogs is the product of the internet age, and its development team is also uncoupled through the internet. All 5 members were known each other for a while by developing Go and we've never met face-to-face. We use IM(QQ) to discuss, use Trello to assigning tasks, and use GitHub to work together. Four of them are taking time after work to join the project except me(as a student), it's remarkable.

Now, please let me introduce our develop team:

  • @Unknown:Project manager, back-end developer
  • @lunny:Back-end Git and database developer
  • @fuxiaohei:Front-end developer
  • @slene:Front and back-end developer
  • @skyblue:Back-end developer

Current Development Status

We've released the first public version with many features but it only has very low version number v0.2.0 and status alpha. I have to say that we do not recommend and Gogs isn't ready for enterprise usage. So does Gogs just something look nice and useless? Of course not! The core of Git hosting is the Git data, Gogs is pretty stable there, so every time new version is released, you only need to do copy-paste binary upgrade with no damage to your Git data.

Getting Started

All documentation is in the GitHub Wiki which helps to give you a deeper understanding of Gogs.

To be more convenient, Gogs also provides an installer page(go to URL:`/install`) to help you configure for the first-time run.

Notice

We're glad you choose Gogs, and here we have some notices for you:

Future Plans

  • In general, we'll release new mirror(+0.1) version every month in average.
  • For non-English users, we plan to support i18n in v0.5.0. Keep eyes on Issue #9.
  • Optimizing CPU and memory usage is big part of future releases.
  • To keep tracking our news, follow us on Twitter.
  • Since we distribute Gogs in binary, gobuild.io is our compile and download service provider.

Thank you for taking time read this post, Gogs will grow much better with your support and feedback!

Go development environment for Vim

By Fatih Arslan

29 March 2014

The reason for creating vim-go

Go has a very versatile toolchain and commands that makes Go programming fun. One of the famous tools is gofmt, which automatically reformats the code according to some predefined rules. However there are many other tools like goimports, oracle, godef, etc.. which help to provide a more productive workflow.

There are many independent vim plugins that integrate these tools into Vim. We also have the offical Vim plugins that provides some basic Go support for Vim. However there are many flaws with these plugins:

  • Gofmt destroy the undo history and it's impossible to do undo on your code
  • Building, running or testing are not integrated, all of these needs additional settings.
  • Binary paths are hard coded
  • Plugins are not using the same format. Each plugin is built differently, some support lookup under the cursor, some do not.
  • Syntax highlighting can't highlight functions
  • Auto completion needs a package and additional settings.
  • Many other small flaws..

The main reason to create vim-go was to integrate all these plugins, fix the flaws and provide a common and seamless experience.

Vim-go features

vim-go improves and supports the following features:

  • Improved Syntax highlighting
  • Auto completion support
  • Integrated and improved snippets support
  • Gofmt on save, keeps cursor position and doesn't break your undo history
  • Go to symbol/declaration
  • Automatically import packages
  • Compile and build package
  • Run quickly your snippet
  • Run tests and see any errors in quick window
  • Lint your code
  • Advanced source analysis tool with oracle
  • Checking for unchecked errors.

vim-go automatically installs all necessary binaries if they are not found in the pre-defined path ~/.vim-go (can be disabled if not desired). It comes with pre-defined sensible settings. Under the hood it uses goimports, oracle, godef, errcheck, gocode and golint. Let's see how we can make use of those tools.

Installation

vim-go is a pathogen compatible bundle. To use it just clone it into your bundle directory:

$ cd ~/.vim/bundle
$ git clone https://github.com/fatih/vim-go.git

Auto completion is enabled by default via <C-x><C-o>, to get real-time completion (completion by type) install YCM:

$ cd ~/.vim/bundle
$ git clone https://github.com/Valloric/YouCompleteMe.git
$ cd YouCompleteMe
$ ./install.sh

Development with vim-go

On first usage it tries to download and install all necessary binaries. Let's see what commands we now can use. Below is a list of commands vim-go supports, most of the commands are improved and new ones were introduced (some of them are just from the official Go plugin).

vim-go uses goimports and reformats whenever you save your file. However you have the option to disable goimports and explicitly import/drop packages:

:Import <path>
:ImportAs <localname> <path>
:Drop <path>

:DisableGoimports
:EnableGoimports

Godoc can be called on any identifier. Just put your cursor under the identifier and call :Godoc, it will open a new view and show the necessary documentation.

:Godoc
:Godoc <identifier>

Godef is one of my favorites tools. It's find the source definition and jumps to that file (go to definition). To use it just put your cursor under a identifier and hit :Godef. It opens a new buffer (just like ctags). You might add the following settings to your vimrc, to open the definitions in vertical, horizontal or new tab with :

au Filetype go nnoremap <leader>v :vsp <CR>:exe "GoDef" <CR>
au Filetype go nnoremap <leader>s :sp <CR>:exe "GoDef"<CR>
au Filetype go nnoremap <leader>t :tab split <CR>:exe "GoDef"<CR>

Building, testing, running are all important steps in development workflow and should be seamless integrated. vim-go has several features that you can use. First check out the build commands:

:make
:GoBuild

:make is the default Vim command to build a project. vim-go integrates it in way that it doesn't produce any binary. That it is really useful because it doesn't pollute your work environment. Any errors are listed in a quickfix window and can be jumped easily with default :cnext and :cprevious. :GoBuild is similar to :make, but it creates a binary for the given main package.

Sometimes we only have small main package that we can want to run and see the output. For that we have:

:GoRun 
:GoRun <expand>

Just calling :GoRun is going to include all files that belongs to the main package (useful for multi file programs). To run a single file just run `:Gorun %`. You can map this to a key, like <leader>r:

au Filetype go nnoremap <leader>r :GoRun %<CR>

To call `go test` just run:

:GoTest

Another tool we have is errcheck, which checks unchecked errors:

:GoErrCheck

Linting is useful to print out mistakes or tips about coding style. For example if you don't provide any documentation comment for a function golint will warn you. To call it just execute:

:Lint

To see the dependencies of your current package run :GoDeps. If you have multiple files you can easily see all source files (test files are excluded) via :Gofiles.

And then we have the still experimental but powerful "oracle" tool. See the extensive official documentation for more info: Oracle docs vim-go implements and includes the following commands (which are part of the offical oracle vim plugin):

:GoOracleDescribe
:GoOracleCallees
:GoOracleCallers
:GoOracleCallgraph
:GoOracleImplements
:GoOracleChannelPeers

These are useful especially if you want to find out how your Code is structured, how your channels are interacting with each other, which struct is implementing which interface, etc...

Summary and Thanks!

Thanks for the following users and projects to make this project happen:

  • Go Authors for offical vim plugins
  • Gocode, Godef, Golint, Oracle, Goimports, Errcheck projects and authors of those projects.
  • Other vim-plugins, thanks for inspiration (vim-golang, go.vim, vim-gocode, vim-godef)

Check out the github page for fare more information (snippets, settings, etc..):

github.com/fatih/vim-go

There is also a Youtube vide that shows vim-go in action:

Youtube: Go development in Vim

There are still tons of modifications and improvements one can make to this setup. vim-go is a new project. Check it out and try it to see how it fits your needs. Any improvements and feedback are welcome.

Plumbing and Semantics: Communication Patterns in Distributed System

By Derek Collison

6 February 2014

Introduction

In distributed systems, multiple components running on different machines—physical or virtual—communicate and coordinate to accomplish tasks.

Distributed systems building typically focuses on the way components communicate. As things change (e.g. infrastructure, configuration, components), pressures arise and must be accommodated. Most of you have heard of loose coupling, which asserts that communications between components should be as flexible and adaptive as possible.

Why is this a design goal? The more rigidly a system is coupled together, the more likely it’ll break down and fail to adapt to change.

Loose coupling is applicable in two main areas: plumbing and semantics. Plumbing is the addressing, discovery, and connectivity between components. Never hard code the IP and port for a communication component within another component's source code. If that machine were to fail and move, it would assume a new IP and port.

For sustainable loose coupling, use a configuration file, change it, and run the component again after the information changes. Well known services like DNS locate and connect components and services.

What if you needed components within a system to scale independently? More than one for a certain component class? This is horizontal scale. Load balancers and proxies help solve the problem and components talk through these intermediaries. DNS can also return multiple entries, and Anycast is also applicable.

What about service-level agreements (SLAs)? How do you maintain high SLAs in fast-changing environments?

Plumbing and Semantics

Plumbing and semantics. Plumbing connects a cell phone call and tunes to your favorite radio station. Semantics are languages people speak to communicate over the phone or radio. Distributed systems usually characterize this by a message payload. These days, many new plumbing and semantics architectures use HTTP for the plumbing and JSON for the semantics, or the “payload”. Both HTTP and JSON are well supported in modern languages and toolkits.

Today, I often use HTTP and JSON in distributed systems. I’ve spent 20+ years designing, building, and utilizing Publish/Subscribe Messaging Systems. They remain a core architectural pattern of any distributed system I design and are used for addressing, discovery, and a flexible control plane. I typically use JSON for the semantics, regardless of whether the plumbing is HTTP or a messaging system.

Within a basic Publish/Subscribe System, messages are sent to topics or subjects, and that—rather than IP sets and ports—upholds addressing in the system. Subjects connect Senders and Receivers. Note that all subscribers matching the subject will receive the message. Simply adding a new subscriber to the system allows multiple recipients to receive a message without further configuration changes.

Many messaging systems also use queueing, a pattern where multiple subscribers exist, but only one receives any given message. I believe that a messaging system should provide the way to have many queue groups in a distributed fashion, and that they should be able to co-exist with normal publish-subscribe operations.

I believe queueing is an interest-based operation, not a sending operation found in many popular queueing products. An interest operation, or something that a subscriber does, enables both normal publish/subscribe and queueing patterns to co-exist in the same subject space and process messages accordingly from a single publisher. This has important implications. A group of subscribers forming a queue group can horizontally scale and reduce overall response time to inbound queries. If queueing were a publish operation, it’d be difficult to add other subscribers or an additional queue group of subscribers to achieve system functionalities such as logging, analytics, or audit trails.

Request/response overlays both publish/subscribe and distributed queueing patterns. A requestor sends a request and receives one or more responses. Again, all connected via subjects, so that requestors and responders move and scale at will without configuring changes and updates. Loose coupling at its finest.

NATS and the associated high-performance server are an open source messaging system built to satisfy these patterns. Unlike traditional messaging systems, it does not have persistence, transactions, or any type of guaranteed delivery models. In my opinion, these were never needed by the industry (though I built quite a few of them), and actually led to bad distributed system design. Think of NATS as a nervous system that simply fires and forgets and presents an ever-present dial tone service that protects itself at all costs. I introduce NATS to provide context for the following examples.

Here’s an applicable example that illustrates the power of these patterns:

Google search is always fast, returning results instantly. Speed has always been a Google mantra. (Disclaimer: I worked at Google from 2003-2009.) How does it work? Google divides widespread information into different buckets or shards. Your request goes to all shards that answer the question based on what they know, then the answers are collected, ordered, delivered to you. Since speed reigns, shards are replicated, all replicas get the request and answer at (about) the same time, and the fastest one back wins. Get the answer to the user as fast as possible!

Example

While not reproducing Google search, we will use similar patterns to achieve a speedy result. A simple adder adds multiple integers together. We can use more adders to represent “shards”, but I leave that as an exercise for you. We replicate the adder classes and identify them with IDs that they generate on startup. Requests will be sent to all replicas, which respond with both the answer and ID of the responders so we see who "wins".

Below is the main loop for the example which can be found on Github.

// Flags
var numResponders int
var numRequests int

flag.IntVar(&numResponders, "numResponders", DefaultResponders, "Responders to spin up.")
flag.IntVar(&numRequests, "numRequests", DefaultRequests, "Requests to send.")
flag.Parse()

// Start the NATS server.
startNatsServer()

// Spin up the appropriate number of responders.
fmt.Printf("\nSpinning up %d responders.\n\n", numResponders)
for i := 0; i < numResponders; i++ {
    adder.NewAdder(ReqSub)
}

// Grab a client connection for sending requests.
nc := adder.NatsConn()

// Time to wait for a response before timing out.
ttl := 10 * time.Millisecond

var req *adder.Request
var resp adder.Response

// Send some requests.
fmt.Printf("\nSending %d requests.\n\n", numRequests)
for i := 0; i < numRequests; i++ {
    req = &adder.Request{X: rand.Int63() % 100, Y: rand.Int63() % 100}
    nc.Request(ReqSub, req, &resp, ttl)
    fmt.Printf("Request: %+v\tResponse: %+v\n", *req, resp)
}

fmt.Printf("\nFinished\n\n")

Interest Graph Pruning

If there are many possible responders, the requestor could face CPU spikes as the client library discards responses 2-N. NATS proactively prunes interest graphs on the fly. Concretely, this means that NATS understands that the client desires one response to the request and therefore prunes the subscription (and hence the interest) after the first response is sent to the client. This alleviates CPU load on the client, and allows the requestor pattern to operate efficiently at scale.

Summary

Modern communication patterns in distributed systems enable speed, scalability, adaptability, and optionality. Many such patterns exist, and we have only touched upon a few in this article. Many projects utilize messaging as a foundation in their systems, such as OpenStack, Baidu, Cloud Foundry, and Continuum by Apcera.

Please feel free to contact me on Twitter, Github, or by e-mail if you have any questions.

See the index for more articles.

GopherCon 2014

Gophercon 2014

GopherCon 2014 Needs Sponsors

Gopher Academy is looking for sponsors for GopherCon 2014. Sponsoring this event is a great way to get your products, services, an ideas in front of top-notch Go developers and decision makers.

Check out our Sponsorship page today and help make this conference amazing.

About Us

Gopher Academy is a group of developers working to educate and promote the #golang community.

Article List

See Article Index


© Gopher Academy 2013