To be honest, as of late I haven't really logged into the FollowWatch account or done much with it the past few weeks because of stressing over getting my taxes done. Well tax day has come and gone, so that's a burden off my back.
Before I mention the tweaks, I wanted to talk about invites for a quick moment. If you're queued for an invite, nothing's going to happen to your place on the queue. You won't be removed. You only run the risk of being removed from the queue if you unfollow the page. Queued users are deleted from the front of the list as invites are sent out to those individuals. When a block of X (could be any amount) invites are being handed out, it automatically goes down the list of people in the queue in order they joined the queue. It generates that person's unique invite, deletes them out of the queue, and attempts to send the invite. DMs can't be sent to people who don't follow you, so if you're not following, that invite fails to send. If that invite fails to send, it's deleted and the process moves on to the next person in the queue until X invites are successfully sent. What this also means is, following/unfollowing/following again won't affect your position in the queue UNLESS the attempt to invite you already came by.
And most importantly, the big reason invites are delayed, twitter's moving way too slow these days, don't want to break the app for the people already signed up.
Regarding the tweaks, made some adjustments to report processing. Most users report more than once a day, but some keep it once a day and stick with the defaulted 12 am eastern report hour as their single hour. Still though, even for most users, their followers don't change in the hundreds daily. But there are a couple users of the application who do. And most of their changes are adds not drops. Leveraging their follower information for their reports reduces the number of calls to twitter which brings speed improvements to the reporting process when there are a lot more users to process reports for in a particular hour. It also increases capacity for number of people that I can allow access into the application. So some more invites should be coming soon.
The other adjustment corrected a bug. Well, not so much a bug, but a nature of the design of the system. After reports are generated, the follow changes recorded (adds/drops) gets deleted from the system. The "bug" occurs when twitter's running sluggish and causes one hour to overlap another, where changes recorded for the current hour were deleting with the changes recorded for the previous hour. Changes have been made to eliminate that conflict.
Thursday, April 16, 2009
Monday, April 13, 2009
Mikeyy Worm
Opened up a test page to check all the vulnerable points to spread the Mikeyy worm (intentionally getting my page affected by it). What I found were:
1. twitter.com/infecteduser
2. twitter.com/infecteduser/followers
3. twitter.com/infecteduser/friends
4. twitter.com/infecteduser/status/any-status-id
5. twitter.com/yourpage/followers?page=any-page (if you are followed by an infected user on that page, with or without "?page=number")
6. twitter.com/yourpage/friends?page=any-page (if you are following an infected user on that page, with or without "?page=number")
And if you are already infected and trying to fix it
7. twitter.com/account/settings
From what I've gathered, twitter appears to have stopped the worm from spreading from points: 1, 5, 6.
2, 3, 4, 7 appear to still be vulnerable.
Unfortunately I was not able to test to see if there were any other points for this worm to spread because Twitter closed the test account I created (for strange activity no less).
As I've posted on both my personal account and the followwatch account, this thing spreads through user's real name. So if you do become infected, you should turn off javascript (so that you won't reinfect on #7 in the list) and update your real name field. Additionally it changes your More Info URL, makes your profile public (unchecks Protect My Updates), and changes your background color to "Mikeyy". To remedy the last one, go to http://twitter.com/account/profile_settings and go into "Change design colors"
This is all the information I have available to assist everyone. Must get some rest right now but hope this information helps.
1. twitter.com/infecteduser
2. twitter.com/infecteduser/followers
3. twitter.com/infecteduser/friends
4. twitter.com/infecteduser/status/any-status-id
5. twitter.com/yourpage/followers?page=any-page (if you are followed by an infected user on that page, with or without "?page=number")
6. twitter.com/yourpage/friends?page=any-page (if you are following an infected user on that page, with or without "?page=number")
And if you are already infected and trying to fix it
7. twitter.com/account/settings
From what I've gathered, twitter appears to have stopped the worm from spreading from points: 1, 5, 6.
2, 3, 4, 7 appear to still be vulnerable.
Unfortunately I was not able to test to see if there were any other points for this worm to spread because Twitter closed the test account I created (for strange activity no less).
As I've posted on both my personal account and the followwatch account, this thing spreads through user's real name. So if you do become infected, you should turn off javascript (so that you won't reinfect on #7 in the list) and update your real name field. Additionally it changes your More Info URL, makes your profile public (unchecks Protect My Updates), and changes your background color to "Mikeyy". To remedy the last one, go to http://twitter.com/account/profile_settings and go into "Change design colors"
This is all the information I have available to assist everyone. Must get some rest right now but hope this information helps.
Friday, April 10, 2009
Login Errors
Some people may have been getting "Invalid Password" errors incorrectly. This issue has been resolved. The twitter library used to verify your credentials used a POST request (i.e. think submitting a web form) instead of a GET request (i.e. think visiting a web page directly) to the verification page. Twitter made updates on Wednesday which prevented requests requiring GET format working under POST format. I wasn't aware the library was using the wrong format.
Monday, April 6, 2009
FW Updates
As of late I haven't done much with FollowWatch. Since the initial 1000 invites ran out, there's been a growing list of people waiting to use the service. A few days into the queue, 200 invites were sent, and a few days later 100 more invites were sent. Given Twitter's recent growing pains, it makes more sense to hold off on giving out more invites as Twitter's stability corresponds with the application's ability to do its job and simply put less users on our end while Twitter works their issues out leads to better stability for Twitter and consequently, better stability for the app.
Today saw some intermittent issues with updates getting sent out. What appears to be the issue either returning an empty or incomplete list of your followers during one stat run and giving the right stats on another stat run. This filled the database with changes like duplicate added/lost followers as well as reporting followers you already had as added followers. Cleared out all that information, and not sure if the API is acting correctly now or still giving trouble.
I read a tweet regarding the invite system that states the following: "Artificial scarcity to improve perceived authority?" First, scarcity is definitely not artificial. We can only query the user information of about 20,000 users in an hour. That means, 20,000 unique names can be looked up that get listed in the gained/dropped column. But even that's not completely true because of Twitter's speed & load issues, which make looking up 4,000 names more often than not take as long as 30 minutes to process. There are real limitations to keep the service flowing and opening the flood gates would quickly debilitate it and make it as unreliable to those using it as Qwitter turned out to be. Secondly, there is no attempt to be perceived as an authority of any sort on follower stats. Whipping up a service of similar fashion to this one shouldn't pose difficult for anyone. It took me a matter of 5 days from concept to site launch, may take you even less.
Finally I don't appreciate people berating me with negative comments because they haven't received an invite yet. This is an add-on service to twitter provided by me as a personal courtesy. This is a hobby not business. And the more you make it feel like business, the more I think about closing the whole queue process off and just inviting people at whim from my personal account.
I also wanted to note: changing your username disrupts your stats if you're already registered, and changing your username before you get an invite from the queue will end up getting you bumped off the queue when your invite is delivered/attempts to get sent out.
Today saw some intermittent issues with updates getting sent out. What appears to be the issue either returning an empty or incomplete list of your followers during one stat run and giving the right stats on another stat run. This filled the database with changes like duplicate added/lost followers as well as reporting followers you already had as added followers. Cleared out all that information, and not sure if the API is acting correctly now or still giving trouble.
I read a tweet regarding the invite system that states the following: "Artificial scarcity to improve perceived authority?" First, scarcity is definitely not artificial. We can only query the user information of about 20,000 users in an hour. That means, 20,000 unique names can be looked up that get listed in the gained/dropped column. But even that's not completely true because of Twitter's speed & load issues, which make looking up 4,000 names more often than not take as long as 30 minutes to process. There are real limitations to keep the service flowing and opening the flood gates would quickly debilitate it and make it as unreliable to those using it as Qwitter turned out to be. Secondly, there is no attempt to be perceived as an authority of any sort on follower stats. Whipping up a service of similar fashion to this one shouldn't pose difficult for anyone. It took me a matter of 5 days from concept to site launch, may take you even less.
Finally I don't appreciate people berating me with negative comments because they haven't received an invite yet. This is an add-on service to twitter provided by me as a personal courtesy. This is a hobby not business. And the more you make it feel like business, the more I think about closing the whole queue process off and just inviting people at whim from my personal account.
I also wanted to note: changing your username disrupts your stats if you're already registered, and changing your username before you get an invite from the queue will end up getting you bumped off the queue when your invite is delivered/attempts to get sent out.
Thursday, April 2, 2009
Server E-mail Issues
Whenever you follow @followwatch, the e-mail gets received by a program which automatically follows you back and sends you an invite (or puts you on the queue and informs you you're on the queue). Unfortunately sometime last night, from the logs approximately 10:15 pm Eastern US, the mail server stopped accepting mail. This is an issue that I'm investigating. I mention this just to say that anyone who followed the page after the time e-mails stopped getting in was not put in the queue. Once the issue is resolved you will be put on the queue, so don't worry about that. I just wanted to keep everyone informed about that.
On a side note, 100 more invites were sent out last night.
On a side note, 100 more invites were sent out last night.
Tuesday, March 31, 2009
Response to a blog comment
My previous blog post received the following comment:
Why offer a service that can't meet the demands? Just curious.
Here is my answer:
I had another paragraph in the post and decided to omit it but perhaps I should have kept it, it may have answered this question before it was asked. Simply put, this service was established as my way of tracking my own followers which I figured I would share with other people once it came to mind. The entire process of establishing the service went from a Saturday afternoon when the idea came to me to launching it 5 days later on a Wednesday morning. That isn't to say I won't do what I can to accommodate those using it and those who want to use it, but the website is more a personal project of mine than a business. Twitter could (but likely won't) defeat the purpose of this app with a single change in which you get a notice when somebody stops following you the same way you get a notice when they start.
Outside of these facts, the limitation to meeting demands is no fault of my own. The service itself does not create a heavy load on the computer that runs it. The limitation to meeting demand is on the side of Twitter's API, which this service is built upon. Enhancements to Twitter's API would open up the service to a larger capacity of people, but as it stands, the level is many times smaller.
Why offer a service that can't meet the demands? Just curious.
Here is my answer:
I had another paragraph in the post and decided to omit it but perhaps I should have kept it, it may have answered this question before it was asked. Simply put, this service was established as my way of tracking my own followers which I figured I would share with other people once it came to mind. The entire process of establishing the service went from a Saturday afternoon when the idea came to me to launching it 5 days later on a Wednesday morning. That isn't to say I won't do what I can to accommodate those using it and those who want to use it, but the website is more a personal project of mine than a business. Twitter could (but likely won't) defeat the purpose of this app with a single change in which you get a notice when somebody stops following you the same way you get a notice when they start.
Outside of these facts, the limitation to meeting demands is no fault of my own. The service itself does not create a heavy load on the computer that runs it. The limitation to meeting demand is on the side of Twitter's API, which this service is built upon. Enhancements to Twitter's API would open up the service to a larger capacity of people, but as it stands, the level is many times smaller.
Where's my invite?
Common question I receive by @reply (or rather by mentions now) and Direct Message is, "When will I receive my invite?"
The short and quick answer is I do not know.
The number of users the application can sustain with its current format is still questionable. It's not the number of users but the number of changes that those users followship undergoes. Very popular users are more likely to inhibit the service than people who undergo follow changes in the single digits. The frequency those popular users get their updates is also a factor. Whether popular users will have to be enforced with the policy of getting their updates more frequently so as to distribute their effect on the app's hourly twitter access is debatable. How that can be done in an automated way is another question.
Another question is, given the state of twitter's API, will it come to a point where some users will have to be given certain days of the week they can get stats so as to allow for more users to use the service? These are all questions that need to be answered before all the people in the queue can be given access to the service.
The short and quick answer is I do not know.
The number of users the application can sustain with its current format is still questionable. It's not the number of users but the number of changes that those users followship undergoes. Very popular users are more likely to inhibit the service than people who undergo follow changes in the single digits. The frequency those popular users get their updates is also a factor. Whether popular users will have to be enforced with the policy of getting their updates more frequently so as to distribute their effect on the app's hourly twitter access is debatable. How that can be done in an automated way is another question.
Another question is, given the state of twitter's API, will it come to a point where some users will have to be given certain days of the week they can get stats so as to allow for more users to use the service? These are all questions that need to be answered before all the people in the queue can be given access to the service.
Friday, March 27, 2009
Latest Information
First, I just want to give acknowledgement to the wonderful people at FHOKE. They are a creative digital agency using the service. They offered, and I gladly accepted, to redesign the follow watch website. So the site has a new layout. The original layout I designed really was meant to be a temporary thing, and FHOKE really came through.
Now on to bugs, updates, invites, and some stats.
Past couple of days some people have been getting empty reports, or reports with empty names.
Example: Lost: ,,,@twitterperson,,@person2,
This is caused by twitter returning an empty name value when the reports get generated. This usually doesn't happen, and most names are properly reported, but the same way twitter the site's been acting up, twitter the API, where FollowWatch gets its data from, is also acting up. An update will be made to remove those listings from your report.
An update will also be made to keep your avatar up-to-date. Currently the one you had when you registered is stored and shown with your updates. This works fine except for users who change their picture. These updates may get posted Saturday.
In other news, the queue of users waiting to sign up has currently reached over 700 people. The app is at about 850 users, so I will probably be sending out about 150 invites. These invites will be automatically sent to people in the order they got on the queue. Once the app reaches 1000 users, it may be at least a week before more people get invited. I have to make sure it can sustain the number of users. Some higher end users may get forced into more frequent updates, or may have to move some users to different time slots.
Current Distribution of Report Generation
12am - 519 reports to generate
1am - 63 reports to generate
2am - 71 reports to generate
3am - 262 reports to generate
4am - 79 reports to generate
5am - 56 reports to generate
6am - 83 reports to generate
7am - 61 reports to generate
8am - 94 reports to generate
9am - 82 reports to generate
10am - 91 reports to generate
11am - 64 reports to generate
12am - 175 reports to generate
1pm - 72 reports to generate
2pm - 87 reports to generate
3pm - 83 reports to generate
4pm - 94 reports to generate
5pm - 78 reports to generate
6pm - 113 reports to generate
7pm - 75 reports to generate
8pm - 104 reports to generate
9pm - 98 reports to generate
10pm - 98 reports to generate
11pm - 72 reports to generate
All times US Eastern
Sunday I will be offline @ the ShowBiz Expo in New York City. You can follow the Expo on twitter http://twitter.com/theshowbizexpo.
Now on to bugs, updates, invites, and some stats.
Past couple of days some people have been getting empty reports, or reports with empty names.
Example: Lost: ,,,@twitterperson,,@person2,
This is caused by twitter returning an empty name value when the reports get generated. This usually doesn't happen, and most names are properly reported, but the same way twitter the site's been acting up, twitter the API, where FollowWatch gets its data from, is also acting up. An update will be made to remove those listings from your report.
An update will also be made to keep your avatar up-to-date. Currently the one you had when you registered is stored and shown with your updates. This works fine except for users who change their picture. These updates may get posted Saturday.
In other news, the queue of users waiting to sign up has currently reached over 700 people. The app is at about 850 users, so I will probably be sending out about 150 invites. These invites will be automatically sent to people in the order they got on the queue. Once the app reaches 1000 users, it may be at least a week before more people get invited. I have to make sure it can sustain the number of users. Some higher end users may get forced into more frequent updates, or may have to move some users to different time slots.
Current Distribution of Report Generation
12am - 519 reports to generate
1am - 63 reports to generate
2am - 71 reports to generate
3am - 262 reports to generate
4am - 79 reports to generate
5am - 56 reports to generate
6am - 83 reports to generate
7am - 61 reports to generate
8am - 94 reports to generate
9am - 82 reports to generate
10am - 91 reports to generate
11am - 64 reports to generate
12am - 175 reports to generate
1pm - 72 reports to generate
2pm - 87 reports to generate
3pm - 83 reports to generate
4pm - 94 reports to generate
5pm - 78 reports to generate
6pm - 113 reports to generate
7pm - 75 reports to generate
8pm - 104 reports to generate
9pm - 98 reports to generate
10pm - 98 reports to generate
11pm - 72 reports to generate
All times US Eastern
Sunday I will be offline @ the ShowBiz Expo in New York City. You can follow the Expo on twitter http://twitter.com/theshowbizexpo.
Wednesday, March 25, 2009
Updates
Ran out of the first 1,000 invites, not everyone used one. Been inviting some people manually (people who wrote @followwatch for an invite and people who DM'd about an invite). Currently at time of writing, tracking stats for 815 twitter users who have over 350,000 followers combined. Updated some stuff on the website earlier today so people understand each of the checkboxes on the sign up page and settings page. Updated some things in the database to support users who have massive added and dropped friends between reports. One user had well over 400 add/removes between reports that failed to get the report stored. The update should fix that.
Once I'm at 1000 users I'm going to hold off on sending invites out until I can make sure the site can sustain the amount of users & their gained/lost followers from a standpoint of Twitter's API limitations.
If you are a current user of the service and only get updated once an hour, please go to the settings page and select at least one more hour. The more frequent reports are updated, the more people the service can support.
Once I'm at 1000 users I'm going to hold off on sending invites out until I can make sure the site can sustain the amount of users & their gained/lost followers from a standpoint of Twitter's API limitations.
If you are a current user of the service and only get updated once an hour, please go to the settings page and select at least one more hour. The more frequent reports are updated, the more people the service can support.
Monday, March 23, 2009
The Process
If this isn't the first site that tracks your un-followers, you may have heard of useqwitter.com. I didn't actually hear about this service until I was 2 days into my development of this service. That site does not require your password, it asks for your username and your e-mail address and it sends you an e-mail when someone unfollows you. At least that's how it's supposed to work. While I decided to continue on with my development of this site, I did sign up with useqwitter. I have not yet received a single e-mail.
This is caused by limited resources. In order to maximize the number of users who can use this service, the number of followers it can track, it leverages some of the twitter data access through your account. Looking up your followers is done through your account. By operating this way, it maximizes the amount of people who can subscribe to this service.
Additionally, at any point you feel like you are dissatisfied or you feel uncomfortable with the site having your data, you can delete it from the system. The last thing I want to do is make anyone feel uncomfortable about sharing this information.
Ultimately I will say this, if you don't trust the app, don't use it. No harm, no bad feelings :)
This is caused by limited resources. In order to maximize the number of users who can use this service, the number of followers it can track, it leverages some of the twitter data access through your account. Looking up your followers is done through your account. By operating this way, it maximizes the amount of people who can subscribe to this service.
Additionally, at any point you feel like you are dissatisfied or you feel uncomfortable with the site having your data, you can delete it from the system. The last thing I want to do is make anyone feel uncomfortable about sharing this information.
Ultimately I will say this, if you don't trust the app, don't use it. No harm, no bad feelings :)
Friday, March 20, 2009
Repeated Names
With the site on its current schedule of daily reports, you may see names in both columns of your report if the person added you and removed you in the same day. The spam_or_APIfail listings appear to definitely be spam accounts caught by Twitter.
This weekend's updates will include excluding these items from your report data.
This weekend's updates will include excluding these items from your report data.
Wednesday, March 18, 2009
Updates
Day one was relatively successful. There were a few bugs found here and there, but they were quickly caught and resolved. One user was removed by request until a feature was added, and two suggestions were made, which lead to another feature request.
The first request was that tweeting your stats to your page be optional. I made a false assumption that people didn't mind this information being made public to their followers. Relatively easy to add in. Probably add in this weekend.
The second request was that the status updates came less frequently, perhaps daily or semi-daily. This has already been changed. Your followers are still processed on an hourly basis, but your report is only updated daily. It's been suggested to allow users to pick which hours of the day the daily reports are processed. This requires a little more effort to change and will probably get added in this weekend as well. Until then, reports are generated for every user once daily at Midnight Pacific Time.
Feedback is always encouraged and any thoughts on these changes or features should be commented.
The first request was that tweeting your stats to your page be optional. I made a false assumption that people didn't mind this information being made public to their followers. Relatively easy to add in. Probably add in this weekend.
The second request was that the status updates came less frequently, perhaps daily or semi-daily. This has already been changed. Your followers are still processed on an hourly basis, but your report is only updated daily. It's been suggested to allow users to pick which hours of the day the daily reports are processed. This requires a little more effort to change and will probably get added in this weekend as well. Until then, reports are generated for every user once daily at Midnight Pacific Time.
Feedback is always encouraged and any thoughts on these changes or features should be commented.
Real Time Polling
A user, @Meezyy, suggests real time polling. It is a nice idea but there's a few hurdles to deal with on this.
The first is not to bog down Twitter with requests. Most people have noticed the past couple of days twitter's been dealing with issues, delayed tweets, delayed direct messages, things appearing sometimes and then disappearing to reappear again later.
The second is not to debilitate our service or your usage of other apps. If you use a client like TwitterBerry, TwitterFon, TwitDroid, etc, you are using API calls. TweetDeck keeps you informed about these. Every account is given 100 API calls per 60 minute timeframe. You use an API call every time you check @replies, direct messages, your personal timeline (twitter.com/yourname), your friends timeline (twitter.com/home), etc. using anything but the twitter.com website. Updating your status doesn't count against this limit. After you use those 100, you're locked out until the time resets. TweetDeck columns won't get refreshed, checking friends/@replies/dms from twitterfon/twitterberry/twitdroid/etc. won't get any new tweets.
FollowWatch polls your followers list once an hour from your account (using 1 of your API calls). It's the reason behind getting your password. Retrieving username info for each of the people that have changed on your friend's list is retrieved through our account. FollowWatch is allowed 20000 calls per 60 minute timeframe as a developer account. While it probably could update in real time by polling more frequently, it wouldn't be able to provide stats for that many people.
Still, if a way existed, it's a feature worth looking into.
The first is not to bog down Twitter with requests. Most people have noticed the past couple of days twitter's been dealing with issues, delayed tweets, delayed direct messages, things appearing sometimes and then disappearing to reappear again later.
The second is not to debilitate our service or your usage of other apps. If you use a client like TwitterBerry, TwitterFon, TwitDroid, etc, you are using API calls. TweetDeck keeps you informed about these. Every account is given 100 API calls per 60 minute timeframe. You use an API call every time you check @replies, direct messages, your personal timeline (twitter.com/yourname), your friends timeline (twitter.com/home), etc. using anything but the twitter.com website. Updating your status doesn't count against this limit. After you use those 100, you're locked out until the time resets. TweetDeck columns won't get refreshed, checking friends/@replies/dms from twitterfon/twitterberry/twitdroid/etc. won't get any new tweets.
FollowWatch polls your followers list once an hour from your account (using 1 of your API calls). It's the reason behind getting your password. Retrieving username info for each of the people that have changed on your friend's list is retrieved through our account. FollowWatch is allowed 20000 calls per 60 minute timeframe as a developer account. While it probably could update in real time by polling more frequently, it wouldn't be able to provide stats for that many people.
Still, if a way existed, it's a feature worth looking into.
FollowWatch Blog
From a concept Saturday afternoon to a launched website Wednesday. It has been a fun process figuring out how exactly I wanted this service to work. For now we are in invite only phase. I have no idea what kind of viral effect may occur when the service actually launches, so 1000 people is the current limit. At the time of this writing I still have some things to finish up before I twitter this link to the friends on my personal twitter. If you're reading this though, it is noted that the site's been launched. Suggestions on how to improve the service may be left on this blog, or tweeted to @followwatch.
For those that do not know me, my name is Pierre Fontenelle. I developed and ran a website called CrushSpot for 3 years until I made a deal with another company.
So as noted, feel free to leave your suggestions on this blog. I will probably write on any updates made to the service on this blog.
For those that do not know me, my name is Pierre Fontenelle. I developed and ran a website called CrushSpot for 3 years until I made a deal with another company.
So as noted, feel free to leave your suggestions on this blog. I will probably write on any updates made to the service on this blog.
Subscribe to:
Posts (Atom)