A grab-bag release

The Populi team put in a late night last night pushing out a grab-bag of features that'll smooth out some common workflows and make all kinds of data simpler to get at. This post also includes some features that we trickled out over the past few weeks.

Facilities

We moved Facilities from the Account bar to its own tab. Accessible to anyone with the Staff role, Facilities contains Locations and Resources. We also moved the Room Vacancies report from Billing to Locations.

Additionally—in fulfillment of a rather popular feature request—we made it a lot easier to find out who's in what dorm room. Our previous solution wasn't really a solution... if you wanted to know, you'd have to guess or stumble upon it while looking at student profiles. Now, whenever you're looking at a Room page, you'll see the current occupants listed. When looking at a Building page, you can select a date and then export a list of who is/was in that building's rooms as of that date. The export looks for the Academic Terms in session as of the date and then finds who's listed in which room for that Term. This is handy for looking up past occupants and checking who's situated where for upcoming terms.

Financial

Financial has a new report and an upgrade to an existing feature...

The Unapplied Payments/Credits report shows, as its name indicates, all of your unapplied payments and credits along with all the relevant details. We've had a few requests for this, and we're really happy to get this one out the door to y'all.

The History tab in Student > Financial now includes voided transactions. What's more, there's also an Export button which lets you create a spreadsheet of the information in the tab.

Academics

The new Course units by student setting in Academics lets faculty manually adjust the credits/hours for individual students on the course roster. This is really handy for non-traditional programs that award differing numbers of earned units to students in the same course—previously, you would have to create a separate course section for each student and change the units at the course level.

In Data Slicer, you can now see what Degree a Specialization falls under whenever you include the Specialization field in a query or report.

Here and there

We rolled out "Likes"—now, if you see something you like on someone's Profile or Course Bulletin Board, Populi News, or course discussions, you can click the heart to let everyone know that you dig it.

In Bookstore, you can now sell invisible items in Point of Sale. Effectively, this allows you to add things like fees, deposits, etc. as "items" in your Bookstore; being invisible, regular online shoppers won't be able to find them—but you can still sell them to students at your cash register.

We exposed more Admissions data via the API.

As is typical of our releases, we've also been fixing bugs, tweaking the interface, and optimizing scripts and servers on the back-end.

What does Twitter do?

Let's ask a few questions about Twitter and figure out what this company is up to.

What is Twitter?

To hear them tell it, it's "a real-time information network that connects you to the latest stories, ideas, opinions and news [sic] about what you find interesting." The you in this case is the Twitter user. A user signs up with an email address, figures out a unique username, maybe enters some personal information, and finds other Twitter accounts to follow. Then he starts tweeting and telling the world about what he's into.

What makes Twitter go?

Money. Duh. However, Twitter doesn't charge its users for its service—one which spans 20 countries and supports 140 million active users and 340 million tweets per day. What's keeping the lights on?

Venture capital, thus far to the tune of 1,160,000,000.00 green American dollars [1]. Venture capital (VC) is a form of private equity that's invested in young, untested, high-risk companies in return for a share of ownership and, presumably, an out-of-proportion payday in the form of an eventual "liquidity event". A considerable portion of software startups receive VC funding in multiple rounds; that is, when the money from early investors runs out, new investors refill the tank. Popular, free services are especially likely to go this route because scaling something to the size of Twitter while also not charging for it creates certain cash flow problems [2].

The solutions to such cash flow problems are the aforementioned liquidity events: the IPO, in which the company's shares are offered and sold to the public; additional VC rounds where earlier investors sell their ownership to later investors; the acquisition, in which the company is sold to another company and the investors' shares become cash or equity in the purchasing company; or profitability, in which the company's assets finally become a viable, ongoing business.

What assets can Twitter turn into a business?

Any number of things can be turned into a viable business, but the long and the short of it is that you need a product that you can sell to customers for more than that product cost you to make it.

Let's review a second: VC-backed software companies usually need VC money because their product is free and they have no way to support their growth. But the investors are getting into it because there's a fat payday in there somewhere. How do you make fat payday kinda money from software that doesn't charge its users?

  1. You could start charging those users for your product.
  2. You could add new stuff to your product and charge for a premium service.
  3. You could start selling ads.

Good luck charging for your previously-free product. If you want to charge users, you'd better do so right out of the gate. Adding new levels to your existing free product stands a better chance, but those paid customers still have to pay for themselves and all the free users; that's a tough price point to hit.

Then there's advertising. Web-based or digital advertising, worldwide, is an $85 billion concern; some $25 billion of that is spent in the U.S. alone. [3] That's just about too good to pass up if you're a software company that needs a way to convert all your users into a solution to your cash flow problem. Of course, in that size a market, there's a lot of competition. You need some compelling features to set your product apart from the rest.

For Twitter, that compelling feature is the frictionless broadcast of interests and information within Twitter's vast ecosystem and their ability to analyze it, target it, and predict what you'll do. The product, of course, is your attention and all that stuff they've learned about you when you voluntarily shared it with them.

In a word, you are the asset they're turning into a business.

How does Twitter sell their product?

Twitter's customers are advertisers, or brands. An advertiser is a company that buys a shot at your attention so you'll buy their stuff. A brand might only be one part of a company that buys a shot at your attention so you'll buy their stuff. Since advertiser is kind of a dirty, untrustworthy word, companies like Twitter favor the word brand. Accordingly, I've coined the ungainly portmanteau brandvertiser to highlight the just, you know, cosmic importance of that distinction.

Twitter hawks three different angles on their product to brandvertisers: Promoted Tweets, Promoted Trends, and Promoted Accounts.

Promoted Accounts let a brandvertiser hire Twitter to help them get more followers—so when the brandvertiser tweets about something, more people will be there to read about it. Twitter does its mightiest to match brandvertiser accounts to people likely to follow them. If you tweet a stream of bile and vitriol against restaurants that serve cheesecake, Twitter will probably not promote @Cheesecake to you; on the other hand, if you tweet incessantly about kitty cats, you'll probably be hearing from @FancyFeast.

Promoted Tweets are tweets a brandvertiser pays to have injected into user timelines and searches. If you search on Twitter for "tomato cages" and The Cheesecake Factory has paid for the word "Tomato", their Promoted Tweet will show up at the top of your search. If you or people you know follow @Cheesecake, it might even pin itself up in your own timeline.

Promoted Trends let a brandvertiser buy a slot among the top Trends. "These trends are featured prominently next to a user's timeline," says Twitter, which means that a Promoted Trend buys a prime little piece of screen real estate. When you click on a Trend, you see a stream of tweets relating to that trend; when a brandvertiser buys a Promoted Trend, the aim is to jumpstart a "conversation" (hopefully favorable to you!) on Twitter and thereby build up awareness and followers.

These three things are different angles on their very compelling ability to target relevant ads at users based on their analysis of everything you've revealed about yourself. You've tweeted about a show you liked. You've tweeted what you're listening to. You've tweeted about lunch. You've tweeted about purchases. You've tweeted about things you want. You've tweeted pictures. You've tweeted responses. You've retweeted stuff others tweeted. You've tweeted about trends. You've linked it to Facebook. You've linked to your blog. You've favorited tweets. And even if you haven't done any of that, you've still followed others who have. Twitter has some pretty solid information about what you're into and what you're up to. So when a brandvertiser sidles up to Twitter's stall in the giant advertising marketplace, Twitter promises not just your likes and interests, but also some credible predictions about your behavior.

The end result is, hopefully, user engagement, which looks something like this.

In the 1980's, Nike figured out how to get you to pay them for a t-shirt with their logo on it, turning you into a walking billboard for their company and products. Twitter's aiming at the same thing: to have you let them fill your timeline—the digital expression of you—with messages from brandvertisers.

So, what is Twitter?

Twitter is actually an advertising company that sells what it collects from you to companies that want you to buy their stuff.

Advertising? So?

Advertising is nothing new. Mass media—newspapers, magazines, radio, television, and the like—subsist on advertising dollars, and always have. Advertisers have always sought out the best ways to target their ads, and so media companies have poured considerable resources into figuring out who's consuming the content they produce. Print publications had some limited information about their readership—where they lived, political leanings, rough income estimates. Radio and television figured out what kind of people were tuning in and when—housewives during daytime, kids in the afternoon, dads around suppertime, the whole family during primetime, parents at late night, weirdos at 2 AM. Of course, this data was somewhat piecemeal, being largely inferred from Nielsen samplings and general demographic studies. It was also completely anonymous.

Early internet advertising was a lightly re-imagined version of broadcast advertising until the rise of social networks facilitated two developments in the '00's. One, data collection grew in scope and sophistication as these sites made "sharing" features available to their users, who could now tell their friends (and volunteer to the watchers) a minutely-detailed view of their location, interests, politics, income, time, and behavior. Two, social networks facilitated the collection of heretofore completely elusive data: relationships, social interactions, and real-time behavior ("Likes" on Facebook, for instance). Now there was hard data on something called "engagement", and advertising companies started selling that.

Despite those advances in collection and targeting, those 25 billion semolians are chasing an incredibly diverted, desensitized, and uninterested sea of eyeballs. In the early days of web advertising, clickthrough rates—the percentage of times an ad was clicked relative to how many times it was shown—wobbled between five and seven percent. Clickthrough rates now do well to hit 0.3%. In an environment like that, it's one thing to plop an ad in front of 30 million pairs of eyes and quite another to credibly promise that 4% of that 30 million will click it and do something to engage with it.

What's not to like about the advertising business model?

Ads are unwelcome, ugly, and noisy. And there are some deeper, interlocking concerns.

There's what they have on you. The information collected about audiences by newspapers, radio, and TV was necessarily anonymous—they simply didn't have the technology or the means to get specific data about specific people. The product they sold to advertisers was aggregated into market segments and not connected to individuals. The information websites collect about us, on the other hand, is not anonymous—it is anonymized. That is, you give them your name and other identifying information [4], volunteer your interests and other delicious information to them... and they sell everything but your name to the brandvertisers. They, of course, still have your name on it all.

So?

There's what they gotta do with that information. In capitalist [5] systems, the one demand of shareholders, markets, and other faceless abstractions is... ever-increasing profits. That demand forces advertising-dependent companies to wring more marketable assets from their users at ever-decreasing costs. One way to do that is evolve the service to encourage you to give them more data about yourself [6]. Another way is to sell access to the data in new ways. Other avenues have surely been contemplated but not tried.

Whatever the case, whatever money Twitter is making off of you now needs to increase by... a lot. They're in the hole by $1.6 billion of invested capital, or about $11.50 per user. They've spent a bunch of  money filling up their warehouse with product. And now they have to move some of that "engagement"—or something else—out the door.

There's the issue of long-term sustainability. You're giving them something valuable—valuable to advertisers, of course, but also valuable to you: thoughts, pictures, correspondence, videos, jokes, laments, and so on. But you're not providing them with the valuable green things that keep them running. In terms of a business relationship, ad-dependent companies have no obligation to you—to your interests, your content, your privacy... for that matter, they're not even required to stay in business.

Some words from Maciej Cegłowski that get at it perfectly:

People upload photos, videos, email, and all kinds of valuable personal information to websites large and small on the assumption that someone there will take appropriate technical measures to safeguard their stuff.

Most of those websites don’t get their revenue from users. Instead, they rely on some form of advertising, or on investor money they receive in return for telling a credible story about future advertising...

...the real problem with this triangular model is that it gives users no visibility into or control over the relationship that ultimately pays for their long-term data storage. If the advertising market collapses, or if storage costs rise unexpectedly, the site might give them little warning before going belly-up...

...the most common way we store important data online right now has shaky foundations.

Beneath it all, there's a contemptible idea of what "social" means. Sites like Twitter and Facebook think of themselves as social media, as reflections of your real life community, as a way to deepen, extend, and re-establish connections with family and friends, and so on. But the point of all that activity—sharing, following, friending, connecting—is to funnel it to brandvertisers that want to sell you something. Your social life, in other words, has no intrinsic value except as a commodity, like crude oil or soybeans.

It doesn't paint a pretty picture—the stores of data they have on you, the unrealistic expectation that it should yield expanding profits, and the brittle trust between the user and service.

Twitter's not the only one.

The Twitter story is hardly unique to Twitter itself. This is the basic outline of every site that gives stuff away to attract users, takes on investors to keep going and growing, finds itself needing to convert all those users into revenue, and so turns to advertising to keep investors happy. Google and Facebook did or are doing the same thing. Lots of other popular, free services find themselves somewhere on this same rickety rollercoaster.

There are three options:

  • Get comfortable with these free services. After all, it is pretty easy to be part of the unengaged 99.7%—for now. Who knows what kind of revenue hoses they'll have to screw on to the spigot later.
  • Pay for what you use. Lots of companies charge for software that helps you work. But almost no one charges for the bread-and-circus stuff that Twitter and Facebook traffic in. At this time, that's a challenge.
  • Read a book. Tell your friend about it over supper. Have a conversation with him.

All three options require this one thing: that your eyes be open to what's going on.

[sic] Serial commas forever.

1 Via Crunchbase.

2 Via the Pinboard Blog.

3 Figures extrapolated from the estimates at WPP.

4 Identifying information includes things like email addresses.

5 Whatever that means anymore.

6 See the ridiculous Google Glass concept video for this notion taken to an extreme.

Populi now has emoji support

After about twenty minutes of work yesterday, we added emoji support to Populi. Emojis are little inline text characters similar to emoticons. First invented and made popular by Japanese mobile phone operators for use in text messages, they have evolved into a more-or-less standard set of characters.

You can use Emojis anywhere "social" in Populi, such as news comments, course discussions, and bulletin boards. They're simple to add—just type colons around an Emoji keyword (i.e. :smiley:) and post your comment. Your emoji will show up right in the text of what you've written. Check out our Emoji cheat sheet to see what you can do.

Canadian privacy laws and cloud computing

First, the disclaimer

What follows is the fruit of our research into Canadian privacy laws as they bear on the use of cloud-based software to store and access personal data as of mid-March, year of our Lord 2012. Of course, we are not lawyers, nor are we Canadian, nor are we from the future. If you have any questions about federal or provincial privacy laws, please seek out legal counsel from north of the 49th.

Now, the fun part: the article!

While most of our customers are located in the United States, we also have a sizable number of Canadian schools using Populi—and many more who are considering us. One thing our Canadian friends ask us is whether they would run afoul of federal or provincial privacy laws by using our service. The laws to which they refer place restrictions on the storage and access of personal information by public institutions—particularly, they stipulate that such institutions must store such data in Canada.

What laws are in play here?

The laws in question are legion and exist at both the federal and provincial level.

The Personal Information Protection and Electronic Documents Act (PIPEDA) of 2000 is a federal data privacy law that governs how private sector business and organizations collect and use personal information. This law makes no requirement about storing personal data in Canada. It is also not binding on provinces if the provinces have in place laws that are substantially similar to PIPEDA. British Columbia, Nova Scotia, and Alberta have each enacted different versions of such "substantially similar" laws.

Alberta's laws make it illegal for a public body or service provider to disclose personal information to an entity that does not have jurisdiction in Alberta. This requirement does not exist for private institutions. More information and an unofficial list of Alberta public institutions can be found at the Government of Alberta website. Spoiler: none of the institutions listed are private colleges. If you're a private college in Alberta, you can join our other Alberta-based customers in using Populi.

British Columbia and Nova Scotia have each enacted laws that go by the names PIPA and FIPPA. In both Provinces, the Personal Information Protection Act (PIPA) governs private bodies; the Freedom of Information and Protection of Privacy Act (FIPPA) governs public bodies.* British Columbia's laws are more strict, so we'll concentrate from here on out on the laws as they exist in that province.

To unsnarl this, we need to understand two key concepts: personal information and public body.

What is personal information?

From British Columbia's definition of it in PIPA, we learn that, in its most restrictive sense...

"Personal information" means information about an identifiable individual and includes employee personal information but does not include

(a) contact information, or

(b) work product information

So student records, financial information, and much of the other information a school would use Populi to manage are all included in this definition. Pretty straightforward.

What is a “public body”?

Less straightforward is the definition of "public body". According to FIPPA, “educational bodies” come under its domain. Many college officials stop reading there and proceed on the assumption that their school can’t use cloud-based services without running afoul of the law.

However, Schedule 1 of that document contains the following:

"Educational body" means

(a) a university as defined in the University Act,

(b) [Repealed 2003-5-19]

(c) Royal Roads University,

(c.1) [Repealed by 2002-35-8]

(d) an institution as defined in the College and Institute Act,

(d.1) the Thompson Rivers University [Added by 2005-17-17]

(e) [Repealed 2004-33-18]

(f) the Open Learning Agency established under the Open Learning Agency Act, [Amended by 1997-52-40]

(g) a board as defined in the School Act, or [Amended by 1997-52-40]

(h) a francophone education authority as defined in the School Act; [Added by 1997-52-40]

Let's look closer at (a) and (d). The University Act applies to the four major universities in British Columbia, as well as any other university “designated as a special purpose, teaching university by the Lieutenant Governor in Council”. Under this law, none of these institutions could use Populi. No one is shedding any tears over this—Populi is not meant for schools with thousands of students like these institutions.

As for the College and Institute Act, it applies to publicly-owned colleges and Provincial institutes. We gather this from requirements in Section 47 that “a pension plan must be provided under the Public Sector Pension Plans Act to employees of an institution” and some clauses in Section 50, to-wit:

Institution is an agent of the government

(1) An institution is for all its purposes an agent of the government and its powers may be exercised only as an agent of the government.

(2) An institution may, in its own name, carry out its powers and duties under this Act and, with the consent of the minister and the Minister of Finance, acquire and dispose of land or buildings.

(3) Despite subsection (2), an institution may lease, or enter into an agreement to lease, land or buildings for a term that ends on or before the end of the fiscal year in which the institution entered into the lease or agreement.

(4) If an institution disposes of land or buildings, it must not spend the proceeds of the disposition without the consent of the minister.

Are you a public institution? Ask yourself these two questions: Are my employees considered government employees? and Do I need to ask the Minister of Finance if I can buy or sell real estate or spend the proceeds from a sale? If you answer No to both questions, you are, in all likelihood, not a public college—and are therefore free to use Populi to manage your school's information.

If you answer Yes to either question, British Columbia probably considers you to be a public college. Therefore, you may not use Populi unless we stored your data on servers in Canada—a complicated endeavor given how much we rely on other cloud-based services (the Amazon cloud, for instance) to provide Populi at our current price.

Postscript

As we said in the disclaimer, we’re not lawyers. Involving as it does various laws and jurisdictions, this is a complex issue. If you do seek legal counsel and can confirm or deny anything we’ve said in this article, at we’d love to hear from you.

Further Reading

British Columbia's Cloud Computing and Privacy FAQ

British Columbia's Office of the Information and Privacy Commissioner (OIPC)

An article on cloud computing from itWorld Canada

*As a supplement to FIPPA, British Columbia has published a guide to cloud computing (PDF) for public bodies; the long and the short of it is that the cloud is off-limits to public bodies in that province.

Killing bugs

Every field bears thorns and thistles. A good farmer makes sure that they don't proliferate and ruin the crop. For that matter, every profession has its own thistles, its own bunch of ankle-biting thorns lurking in the wheat and the corn. For us here at Populi, those are software bugs, errors in the code we've written that cause functional problems for our users. So an important part of our job here is finding them and killing them.

Software bugs get their name from the days of computing yore* when a modest computer filled a room with hardware—lockers, tubes, relays, engineers, switches—that moved information from Point A to Point B. "Bugs" in the system were just that: little creatures that got in the works and prevented the physical passage of electrical impulses from one locker or tube to another, thereby preventing the computer from working properly. Troubleshooting those problems had to be, I dunno, just a ton of fun.

Having built Populi without using lockers, tubes, and the like, our extermination system is much less arduous. We start with software testing, or, as it's called in the biz, "Quality Assurance". When our developers deploy code in our local development environment, it's kicked over to our QA crew—Toby and (more informally) Brendan and Isaac. These three run the new feature or function through its paces to determine a few things:

  • Does it work as intended?
  • Does it even work in the first place?
  • Does it break anything else?

We ask other questions during the QA process,** but these three are key. If the QA guys find that the code or feature fails on one of these points, they send it back to the developer who authored it and tell them where to spray for pests. The respective teams repeat this process until the code or feature works. Only then do we release it.

Once we release, our users subject our code to a different kind of testing—that of everyday use in a live environment with all of its attendant surprises and unexpected side-roads. And if something's not working, they let us know.

The first thing we do in our bug hunt is verify whether there's a bug at all. Sometimes the user's workflow has gotten something out of order. Sometimes another user has changed or deleted something the first user was looking for. And sometimes we discover that everything's working fine, but we don't currently accommodate a particular "edge case" that the user needs Populi to handle. If it's the first problem a little garden-variety training is in order; if the second, we update the feature to handle the "edge case".

But if the user's problem turns out to be neither of the above, it's time to start digging through the code to see what grubs and creepy-crawlies we discover.

The digging is the part that's most like the old days of checking each switch, tube, and locker for moths. One of our programmers—usually Patrick—simply reads through the code, line-by-line, to find what's wrong. A few things to keep in mind about this job:

  • Computers and software are very literal; they require extremely precise instructions in order to work.
  • Bugs take several forms. The code might have a syntax problem, or is missing a statement, or has a slash where it should have a bracket (among a zillion other things).
  • Indeed, bugs are usually miniscule, but a tiny error is all it takes to transform a precise instruction into nonsense.***
  • Every part of Populi talks to other parts of Populi. And even those functions that aren't directly connected are linked together like that Six Degrees of Kevin Bacon**** game from the late '90's. A bug in one place can affect a dozen other functions that rely on the broken function to do their own jobs.
  • Populi has about 361,000 lines of code.

Of course, knowing the nature of the problem narrows the needle-in-a-haystack odds in the bug hunt. For instance, if someone's having trouble in Admissions, we know not to look at the Library codebase. Of course, given Populi's intra-connectivity, sometimes we do need to look afield for the bug; a problem with Student Billing might trace back to an error in course enrollment, for example.

Whatever the case, a review of the code turns up one or more culprits, at which point we take the next steps—polishing the code, running it through our QA battery, and then finally releasing it to our live servers.via http://ibc.lynxeds.com/photo/bearded-tachuri-polystictus-pectoralis/pampa-tiny-eating-female-giving-moth-one-their-three-ch

Fixing bugs can be very time-consuming, but we're laser-focused on it. We know that the only thing worse than us having to find a bug is having a bug prevent you from getting your work done. For us, bug-killing isn't an abstract matter of "we'll-get-to-that-someday" software development. It's a matter of active, responsive, "get-it-done-yesterday" customer support.

*Yeah, I know, this is over-simplified.

**Especially the question: does it work in Internet Explorer? That browser is a real turkey sometimes.

***We recently found an error that consisted of one wrong character. Easiest fix ever!

****Isaac Grauke once met Kevin Bacon; therefore, there is only one degree of separation between any given Populi feature and the man himself.

New release: SMS Emergency Notifications

We're pleased to release one of our customers' most-requested features: SMS Notifications! 

It's really simple:

First, your users opt-in to receive emergency notifications via SMS in their personal settings. Once they've submitted their verification code, they can now receive SMS messages from you via Populi.

All you need to do is go to the new Emergency Notifications tab in Communications. Compose a message, hit send, and it'll go out to all of your active faculty, staff, students, and advisors who've opted-in (you can also select specific campuses). If any of the recipients haven't opted-in to receive SMS messages, they'll get an email instead.

To help ensure your recipients can continue to get emergency notifications, we lock in their SMS number on their profile so it can't be changed by anyone, not even Staff.

Each of our three Pricing Plans comes with a certain number of SMS messages included for free. Small gets 500, Medium gets 1,000, and Large gets 2,000. If you run over your free allotment, additional SMS are just two cents per SMS recipient.