Improve SI bus service with a hackathon

[IAlam]

My Friend sent this to me and I am interested any tips or useful info you can give me?

[DetSMART45]

Sounds interesting, but they're pretty unclear about any actual goals or what type of background someone attending should have.  Coding for apps is different than other IT specialties.  Fishing expedition, IMO.

 

Since it's open to the public, it'd be an opportunity to get your hands on some data sets that may be insightful to SI bus use overall.  I find it a bit odd in the wording, "by applying technology to ridership and performance data that will be released to the public for the first time."  That could mean more drill-down technical data, but on the surface it raises the flag that those data sets -- which should supposedly be the same they put out in the route-by-route reports already -- aren't so similar.

 

Interestingly, if you go to the GoogleDocs registration page, photo is from these forums (kinda cool, when they could have just used MTA stock photos).

[IAlam]

Sounds interesting, but they're pretty unclear about any actual goals or what type of background someone attending should have.  Coding for apps is different than other IT specialties.  Fishing expedition, IMO.

 

Since it's open to the public, it'd be an opportunity to get your hands on some data sets that may be insightful to SI bus use overall.  I find it a bit odd in the wording, "by applying technology to ridership and performance data that will be released to the public for the first time."  That could mean more drill-down technical data, but on the surface it raises the flag that those data sets -- which should supposedly be the same they put out in the route-by-route reports already -- aren't so similar.

 

Interestingly, if you go to the GoogleDocs registration page, photo is from these forums (kinda cool, when they could have just used MTA stock photos).

I actually emailed them for more information on it I have yet to be given a proper response

The photo credit is actually linked to this website

[peacemak3r]

A lot of Hackathons I've been to don't really have a requirement in terms of skill in coding, they've welcomed every level to join. But of course, the higher level of your coding, the higher chances you'd probably been noticed. Though unless your idea is phenomenal but your coding lacks, I think you'd still have a shot.

[JerBear]

I have paid attention whenever I read about things like this because of this widespread belief that technology is naturally suited to improving things. They all end the same, falling short of any real improvement. In reality, all the online dictionaries and media outlets can't compete with YouTube cat videos and porn. Technology will still, in the end, be about as useful as anything else. Without solid ideas that exist regardless of technology and the political will to execute them, technology is useless. So my prediction is someone tries to spin it afterwards as having created some interesting ideas worth pursuing, but in the end no actual ideas/strategies/decisions in the final published Staten Island Comprehensive Bus Study will have come out of this hackathon.

There won't be some crazy interesting app created that doesn't already exist. They won't discover that by eliminating these two stops, suddenly this route will save 15 minutes.

What we hope comes out of it is enough information sharing and enough of a benefit to the MTA that this pushes them to be more open with data in the future. That's my best hope for this event.

[DetSMART45]

I have paid attention whenever I read about things like this because of this widespread belief that technology is naturally suited to improving things. They all end the same, falling short of any real improvement. In reality, all the online dictionaries and media outlets can't compete with YouTube cat videos and porn. Technology will still, in the end, be about as useful as anything else. Without solid ideas that exist regardless of technology and the political will to execute them, technology is useless. So my prediction is someone tries to spin it afterwards as having created some interesting ideas worth pursuing, but in the end no actual ideas/strategies/decisions in the final published Staten Island Comprehensive Bus Study will have come out of this hackathon.

There won't be some crazy interesting app created that doesn't already exist. They won't discover that by eliminating these two stops, suddenly this route will save 15 minutes.

What we hope comes out of it is enough information sharing and enough of a benefit to the MTA that this pushes them to be more open with data in the future. That's my best hope for this event.

Ahhh, but the problem remains:  Developers come up with ideas, yet the MTA doesn't want to look like dunderheads (since they've got the most "super" people on staff -- which everybody's all paying for.

 

Pretty smart, really: Get someone outside the agency to solve the problems (or give "insight") and then act like it's a collaboration/community involvement.

 

The MTA, like any other government agency, believes only THEY have solutions.  If they get "shown-up", that's a bad thing -- even if it turns out to be good for everyone.

 

Look at all the apps out there for transit which really don't have anything to do with the MTA.  I'm sure a bunch of them are more accurate than the MTA's BusTime and associated apps.  The MTA could, quite simply, turn a lot of online development over to "open-source"/community users and just say, "Hey ... we love those who take interest in this, and this saves money that we can put into improving service for everyone, let's let them do this instead of us doing it in-house."  But that would mean admitting defeat, to an extent, and that can never happen.

 

I can get the BEST, accurate TV listings for my cable system, local broadcast channels, and even competing cable systems from an app that asks for nothing.  If I go to Comcast, AT&T, DirecTV, or even TV Guide (whoda thunk they're still in business), the listings are wrong.

 

But far be it from those players to turn that over to someone who basically gets the same info, from the same sources, but gets it right.  Can't have it look like they're incompetent somehow.

 

This is why I called it a fishing expedition.  The little guys will come up with improvements (hauling the nets in with the fish and processing them) and the MTA will "supervise" and skim the best off (congratulating themselves on a great catch).  Simple as that, IMO.

[checkmatechamp13]

Since it's open to the public, it'd be an opportunity to get your hands on some data sets that may be insightful to SI bus use overall.  I find it a bit odd in the wording, "by applying technology to ridership and performance data that will be released to the public for the first time."  That could mean more drill-down technical data, but on the surface it raises the flag that those data sets -- which should supposedly be the same they put out in the route-by-route reports already -- aren't so similar.

 

I would think that the data would be available by filing a FOIL request (as opposed to some completely top-secret data).

 

They already release the ridership numbers, and they release cost-per-passenger figures whenever they need to cut service.

 

I'm thinking on/off data for all the stops in the network, maybe some form of an origin-destination survey (either conducted from the bus itself or one of those telephone/mail ones). 

[bobtehpanda]

I would think that the data would be available by filing a FOIL request (as opposed to some completely top-secret data).

 

They already release the ridership numbers, and they release cost-per-passenger figures whenever they need to cut service.

 

I'm thinking on/off data for all the stops in the network, maybe some form of an origin-destination survey (either conducted from the bus itself or one of those telephone/mail ones). 

 

Surveys in general are probably not what they're going for, since sample bias in these days is very hard to overcome (not many of the younger folk actually pay for a landline, not everyone has access to the internet or a smartphone, etc.)

 

Depending on how fine-grained data is, perhaps it would be possible to figure out which trips are consistently poor-performing all the time on a bus route; with negative performance consistently for the same route and start time over the course of a year.

[RailRunRob]

My Friend sent this to me and I am interested any tips or useful info you can give me?

Awesome stuff.. Hackatons are a great way's to find new ways to use data and new idea's for near free. Ive been to afew for companies like, Google, Spotify and gracenote they came up with creative new ways to use API's and Audio Fingerprinting. Afterall hacking is all about getting things to do task's it wasn't designed and intended to do. Want to look more into this thanks for the info!

[checkmatechamp13]

Surveys in general are probably not what they're going for, since sample bias in these days is very hard to overcome (not many of the younger folk actually pay for a landline, not everyone has access to the internet or a smartphone, etc.)

 

Depending on how fine-grained data is, perhaps it would be possible to figure out which trips are consistently poor-performing all the time on a bus route; with negative performance consistently for the same route and start time over the course of a year.

 

Yeah, the MTA put up a Powerpoint showing how they're able to use the BusTime data internally, and they mentioned that they can group it according to various standards (they have one of those distance-time graphs that shows how long each run takes to get through each segment of the route. Then they mentioned stuff about being able to zoom into specific segments at certain times. I remember they gave the example of the B63 being late in the PM rush in Sunset Park, while the X17 being early on the northern portion during middays). 

 

They said they can also group it according to the operators (running early, on-time or late), but I doubt they'll release that information, since there's nothing the participants can make of that. 

[bobtehpanda]

Yeah, the MTA put up a Powerpoint showing how they're able to use the BusTime data internally, and they mentioned that they can group it according to various standards (they have one of those distance-time graphs that shows how long each run takes to get through each segment of the route. Then they mentioned stuff about being able to zoom into specific segments at certain times. I remember they gave the example of the B63 being late in the PM rush in Sunset Park, while the X17 being early on the northern portion during middays). 

 

They said they can also group it according to the operators (running early, on-time or late), but I doubt they'll release that information, since there's nothing the participants can make of that. 

 

You would be surprised. At a hackathon I went to last year, the winning project managed to show correlations between personal/criminal history and performance. Apparently the best analysts are all divorcees with lots of speeding tickets.

[IAlam]

I'm curious though this is my first hackathon and all I know is a little C++ I curious on how this will work and what I will have to do, based on what they said it not very specific.

[checkmatechamp13]

I'm curious though this is my first hackathon and all I know is a little C++ I curious on how this will work and what I will have to do, based on what they said it not very specific.

 

I'm in the same boat too. I would assume you could pair up with somebody who knows programming better. I would assume that if you have an idea of what types of statistics to look out for, somebody will be able to generate the appropriate graphs/charts to make the proposals off of.

[IAlam]

I'm in the same boat too. I would assume you could pair up with somebody who knows programming better. I would assume that if you have an idea of what types of statistics to look out for, somebody will be able to generate the appropriate graphs/charts to make the proposals off of.

Form the times i've been to SI and on these forums I've definitely generated a few ideas 

[JerBear]

You kids and your computer languages... Making me feel old...

As part of the North East Queens Comprehensive Bus Study, they analyzed early departures on express buses. But they didn't seem to make any other use of Bustime data. What are the odds that the data is really shallow and doesn't have much information to it that can be useful?

[DetSMART45]

You kids and your computer languages... Making me feel old...

As part of the North East Queens Comprehensive Bus Study, they analyzed early departures on express buses. But they didn't seem to make any other use of Bustime data. What are the odds that the data is really shallow and doesn't have much information to it that can be useful?

That's probably more of a function of a portion of the software's data capabilities that isn't "authorized" -- i.e. paid for to use and analyze things.

 

Or they don't have "enough" people to analyze them.  Hence, this event.

 

Still skeptical.

 

As I've said before, any other company expends money on R&D, because it's part of their edge/survival against the competition, and is a budgeted item.  Any MTA/DOT "study" always seems to hang on having EXTRA money thrown towards it in order to be completed.  Technology is now there ... what's the excuse going to be?

[IAlam]

Just an update

The hackathon is this Saturday. Unfortunately I got an email saying that since I'm under 18 I can't participate. I did get an email however, with the information for the hackathon. I'm not sure if I am allowed to share if but I will as soon as I can so you can review it for yourselves.  

[bobtehpanda]

That's probably more of a function of a portion of the software's data capabilities that isn't "authorized" -- i.e. paid for to use and analyze things.

 

Or they don't have "enough" people to analyze them.  Hence, this event.

 

Still skeptical.

 

As I've said before, any other company expends money on R&D, because it's part of their edge/survival against the competition, and is a budgeted item.  Any MTA/DOT "study" always seems to hang on having EXTRA money thrown towards it in order to be completed.  Technology is now there ... what's the excuse going to be?

 

The notion of "open data" is not really a new thing. Governments around the world at various levels have figured that they collect more data than they possibly know what to do with, so now they just dump anonymized data to the public so that developers can eat their hearts out and make it somehow useful.

 

Companies also organize hackathons with their data as well, so this "R&D" problem is not really exclusive to either MTA or the public sector at large.

[checkmatechamp13]

I will say this: So far this SI Bus Study seems to be going better than the Queens Study. For starters, they didn't have any type of hackathon for Queens as far as I know. I'm not sure how smart it is to give away too much info before the event (since I'm participating), but I will say that working with the MTA representatives at the meetings, they do seem to have a grasp on which particular issues are worth looking into, and which direction they want to move the study in. So make of that what you will as it relates to the hackathon.

[checkmatechamp13]

Here's some of the data they uploaded: bit.ly/mtacartodbdata

 

And some more: http://bit.ly/SIBusData

 

It seems like they placed a lot of emphasis on the coding aspect, rather than actually formulating proposals (so people wrote codes to see delays at bus stops and so forth). In case anybody's interested, here's my proposal.

[DetSMART45]

Here's some of the data they uploaded: bit.ly/mtacartodbdata

 

And some more: http://bit.ly/SIBusData

 

It seems like they placed a lot of emphasis on the coding aspect, rather than actually formulating proposals (so people wrote codes to see delays at bus stops and so forth). In case anybody's interested, here's my proposal.

Don't want to do a TL;DR on you, but just in a quick overview of your CartoDB map, looks like you've got some significant changes in mind.  Am I missing something in a discussion of your proposed ideas, or is that something you're working on?

 

Will be looking at some of that MTA data, just to see what's there, and just how much detail is available.

[IAlam]

Here's some of the data they uploaded: bit.ly/mtacartodbdata

 

And some more: http://bit.ly/SIBusData

 

It seems like they placed a lot of emphasis on the coding aspect, rather than actually formulating proposals (so people wrote codes to see delays at bus stops and so forth). In case anybody's interested, here's my proposal.

oops I forgot to do it myself, but thanks for doing so  

[checkmatechamp13]

Don't want to do a TL;DR on you, but just in a quick overview of your CartoDB map, looks like you've got some significant changes in mind.  Am I missing something in a discussion of your proposed ideas, or is that something you're working on?

 

Will be looking at some of that MTA data, just to see what's there, and just how much detail is available.

 

I did a brief overview of some of the principles that I used in my plan, but since I only had a few minutes left to make a submission, I couldn't go into too much detail.

 

For anybody who's interested, here's the rest of the submissions. 

 

oops I forgot to do it myself, but thanks for doing so  

 

No problem. 

[IAlam]

Anyway since I was told last second was no longer able to attend how was the experience?  

[checkmatechamp13]

 

Anyway since I was told last second was no longer able to attend how was the experience?  

 

It was interesting. Apparently, it was different from other Hackathons in the sense that they gave us the data in advance and told us the exact details of the criteria for judging in advance.

 

Basically, we came in the morning, signed in, had some bagels & coffee and other refreshments. Then the sponsors of the event, and the SI Borough President (James Oddo) spoke. Then they laid out the criteria for judging and gave us the chance to break up into teams (I worked by myself)

 

I'm not good with programming, so I asked the people from CartoDB a lot of questions about the different functions in the map (importing the shapefiles of the routes, etc). They gave us lunch and had a keynote speaker talk about one of the previous Hackathons where they analyzed taxi data.

 

Then at 3:30, they had us submit the proposals, and we had like 2-3 minutes to make a presentation of our findings and proposals. The judges went over the proposals and announced the prizes, and then we went out for drinks (paid for by the sponsors).

 

BTW, for anybody who's interested, here were the criteria used in judging the proposals:

 

Judging Criteria

Submissions will be judged based on the quality of their proposals that leverage data provided at the Hackathon to develop a cost-efficient, equitable, and usable recommendation for the local and express bus systems of Staten Island.

 

Quality of idea

- More passengers benefiting from lower wait times and/or lower travel times

- Fewer passengers experience higher wait times

- Increase in residents and industry/office/retail within service area

- Increase in subway connectivity

 

Implementability

- Cost neutral

- Low number of passengers with negative tradeoffs

- Low loss of geographic coverage

- Low loss of span of service coverage

 

Cost-Effectiveness

- Extra dollars per global passenger-minutes saved (or even dollars saved)

- Extra dollars per additional resident in service area

- Extra dollars per additional industry/office/retail square feet in service area

 

Equitability

- Distribution of geographic beneficiaries, measured by # of census tracts benefiting or with negative tradeoffs

- Distribution of demographic beneficiaries, measured by # of “minority or low income” census tracts benefiting or with negative tradeoffs

 

Staten Island Bus Overview

·      31 local bus lines with 95,000 average weekday riders: Comprehensive intra-borough network with most routes designed to feed into the St. George Ferry Terminal

·      20 express bus lines with 34,500 average weekday riders: Premium fare commuter routes to Manhattan operating with coach-style long distance buses

·      Most popular local bus: S79 SBS between the Staten Island Mall and Bay Ridge, Brooklyn

·      Most popular express bus: X1 between Eltingville and Midtown Manhattan

 

Bus Planning Concepts Primer

·      Generally the longer a bus route is the more unreliable its performance

·      Bus stop spacing: Frequent stops make access to the bus easier but hurt travel speeds and reliability

·      The need to transfer from one bus line to another can be perceived as inconvenient but transfers enable many different destinations to be accessed from many different origins

·      The easiest areas to operate effective bus service have high population densities with well-connected street grid networks

·      Traffic congestion is one of the biggest factors in bus reliability

 

Hackathon Objective

·      Problem statement: Redesign the Staten Island-to-Manhattan express bus network or Staten Island local bus network while keeping in mind:

o   Accessibility: Ease of accessing a bus stop from home and reaching desired destinations

o   Reliability: Ability to operate predictable, evenly-spaced service

o   Speed: Origin to destination travel times that make taking the bus an attractive option

·      Plans should aim to use similar resources (route miles, number of buses required, etc.) to the existing networks.  Plans that increase operating cost or require additional buses for the fleet are less feasible.

 

Express Bus Issues

·      Express network is complex with 20 routes, some of which have multiple variants

·      Many commuters feel they have to take a “tour of the Island” before going to Manhattan

·      Around 36% of express bus commuters drive or are dropped off at the bus stop on Staten Island

·      Express lines often spend considerable time running on heavily congested Manhattan surface streets

·      Around 26% of express bus commuters transfer to a subway line in Manhattan

·      The express bus network is primarily oriented toward areas that do not have quick local bus service to St. George to access the free Staten Island Ferry into Manhattan

·      In many other commuter bus and commuter rail systems, typically, one or a few major terminals in the city center are served rather than try to provide door-to-door service. Would this work better?

·      The X23 and X24 are operated by a private carrier (Academy Bus) but should still be considered

·      When the Gowanus Expressway and Verrazano Bridge construction projects are completed, there will be a seamless HOV/bus lane from Staten Island into lower Manhattan

·      Reconfiguring express buses is likely more easily realizable than reconfiguring local buses as riders are less concerned about what route the non-stop segment uses as long as it gets them to Manhattan fast

 

Potential Express Bus Starting Points

·      The Hylan Boulevard corridor has 8 different express lines.  Could they be condensed into a more simple and frequent network that is more effective and easier to understand?

·      The X17 has 3 variants and very circuitous routing.  How can it be improved?

·      The X21 was introduced in Fall 2014 using limited stops and direct routing to provide a faster trip into Manhattan that connects to north-south transit by running the length of 42nd Street.  Ridership has grown steadily since its introduction.  Could this concept be applied throughout Staten Island?

·      Are certain Manhattan streets better than others in terms of reliability, speed, and customer demand?

 

Local Bus Issues

·      Travel within Staten Island can be inconvenient, particularly from North Shore to South Shore

·      Staten Island has a limited number of continuous roads that enable bus service to connect different parts of the borough, particularly through the Greenbelt

·      Low density, high car ownership, and lack of sidewalks in some areas make bus service challenging

·      Some routes in the network could potentially be restructured to better reflect recent shifts in population and development patterns

·      Keep key destinations in mind: Shopping centers, hospitals, schools, dense neighborhoods, etc.

 

Potential Local Bus Starting Points

·      Port Richmond Terminal (ferry terminal no longer exists): should 4 routes still terminate there?

·      S78 and S74: poor performance, two longest bus routes in New York City. Can they be split apart?

·      S55 and S56: low ridership, circuitous routing.  Can they be restructured to serve the area better?

·      College of Staten Island: Is there an opportunity for more service, particularly from the South Shore?

·      S46/96: Lots of turns due to disconnected road network.  Can the route be streamlined?

·      The S42 and S66 serve riders at the top of two steep hills (New Brighton and Grymes Hill) but do not run on weekends.  Could these routes be restructured with other routes to be more effective on weekdays and make weekend service more feasible?

·      Can the local bus network be better integrated with the Staten Island Railway?