Recent ratings issues

[Ryan Douglas]

I have purged flexible's votes. Sometimes "ratings abuse" is a difficult thing to measure, but recent activity definitely qualifies. I have PM'd him and told him to stop. I recognize he is trying to draw attention to what he perceives as a serious problem, but we disagree that the solution is as simple as what he proposes.

We know some of you care very deeply about the ratings, and we understand why. Some of you do really great work, and invest a whole lot of your personal time doing it. It feels good to be recognized for high quality, and I'm sure it motivates you to keep improving and trying to top yourself in order to truly earn great feedback. On the other hand, we know some of you care not a whit about the ratings, and we understand that reasoning, too. You model because you enjoy it as an end unto itself, and may only value feedback from a handful of people whose opinions you respect. The rest just represents a whole lot of noise.

I know it isn't evident outside of our office, but here at KE we have actually put considerable thought into how we might improve the ratings system. We have discussed many different ideas, from minor tweaks to major revisions which would essentially institute a brand new system.

A looming problem with any of the larger changes we've discussed is what to do with the large body of current ratings. There might be an obvious way to convert them, but depending on the new design, there might not. Those ratings represent a lot of careful thought (and some not so much). Starting everything over at zero, while cathartic, would probably not be well received and is best avoided. Any design we've considered has its own problems; no system is perfect.

Some recommendations I've seen from the forums include: requiring a comment to go with any vote, but allowing anonymity
requiring a name for every vote, but making a comment optional
requiring both a name and a commentBut even amongst those members who don't like the current system there is some disagreement about how it should work instead.

When we consider options such as these, we invariably come to the conclusion that we could make those kinds of changes but that it wouldn't do a lot of good. If comments are required, we'll have a lot of "asdfasdfasdfasdf" and worse from people who don't want to leave one but are now forced. If there's a name to go with every vote, people who feel they received an unfair vote have a lot more ability to retaliate, whether by return votes (if possible) or harassing PMs or...? That's only going to reduce the amount of feedback people receive on their contributions--or it might weed out the lower votes because people don't want to bother posting negative feedback (or feedback they do not believe is negative but which they are afraid will be *perceived* as negative), leaving only praise and thereby artificially inflating the ratings. In all of the above scenarios, we will also have people creating extra accounts in an effort to avoid negative consequences, more so than happens currently.

All of the outcomes described above represent more work for us. They create new problems. Whether we have to start responding to complaints about garbage comments or responding to complaints about members threatening each other or responding to duplicate accounts, we believe we would just be scooting the problem around from one area to another but actually solving nothing. Worse, it would require more of our time than what we have in place now. First we have to implement the change. Then we have to deal with the increased moderation load. Contrast that with the current system that nobody really likes but where the flaws are well understood. People (more or less) can deal with a small amount of bad ratings behavior as long as it doesn't rise above a certain level, and in particularly bad cases of abuse we can step in and intervene.

We understand why many of you don't like the current system. We agree with you on that part. We just haven't come up with anything we think is superior enough to replace it. While some of you are quite confident you possess the ultimate solution, so far we haven't found anything we think qualifies.

We do welcome your suggestions, however, and we do appreciate the feedback. Currently we are not in a position to invest resources in the ratings system. We are quite tied up with other things. Any changes would simply have to wait until later. It is something we are willing to revisit, though.

In the meantime, flexible's blanket 10s have been eliminated, which should restore some normalcy. I'll try to check in over the weekend, but I can't guarantee I'll be able. Please be patient with us if problems persist.

[jeffpn]

Ryan, thank you very much for responding to our concerns. Please erase my ratings (every one I've ever made, if it's easier), as well. I can appreciate the dilemma that you are in. I am willing to help, if at all possible. If there's anything I can do, please let me know.

[Madratter]

While I would prefer requiring a name and a comment, here is another possible suggestion. Allow modelers to simply opt out of the rating system.

That way those of us who think the current system is not just irretrievably broken but a disincentive to doing anything, can simply opt and not have to worry about someone giving us a low rating on plane we just put in 200 hours of work when they cannot even change the size of the prop.

[Boof69]

Not requiring a name for every vote is simply siding with the trouble makers. It's tantamount to me knowing who robbed your house but I won't do anything about it and you can never know who he is. This just allows the problem to perpetuate.
So let me get this straight let the wrongdoers do what they do but prevent the content providers any rights. Sounds like the american justice system as someone texted me earlier. Plain and simple: require a name and much of the problem goes away. If they had the stones to rate like that and be known they'd have done that on their own.

[Maj. Numbskully]

For various reasons I still say anonymity is the root of all evil

lets face it ................long before 10 great !
EVERYONE one handed out 10s like candy
and thats more on us contributors and forum members than any one else...................... no matter what KE will never be able to control that !
we all need a little more restraint:p:rolleyes:

I agree that no matter what or how the ratings work
there will always be some immature *** that will find a way to annoy people

No..I and probably others never did consider that SFDHWHrh type comments that would be left behind , instead of something constructive.......good point
Thanks for explaining KEs ideas and thoughts on this issue Ryan

[jeffpn]

No..I and probably others never did consider that SFDHWHrh type comments that would be left behind , instead of something constructive.......good point

I was talking to Boof yesterday or the day before, and mentioned that that could happen. I would expect it in the beginning, but I think it would die off eventually.

I still think identification would solve more problems than it would cause, but it's KE's forums. It's their right to do it how they choose. Ratings have never been a big deal to me, anyway. I just saw this as an opportunity to try suggest a way to make the system less broke.

[andy29847]

I have been giving 10s on EAs. I do it to try and express appreciation for the work that goes into every model.


It is difficult to rate the variety of models we have fairly. Some of the scale models represent tremendous amounts of research and work and the work that is posted is museum quality. Some of the EAs of actual RC models are very simple, but still achieve the goals of simulating the real RC model.


There could be a rating for each facet of the model - how it flies, the CS, the physical model. That way, an EA could be as high as 30. Postings for AVs or CSs would top out at 10. This would allow the ratings on the swap pages to be more meaningful.

[jeffpn]

If there is to be a rating system, in my opinion it should not be to express appreciation for a model. It should be to evaluate a model in all respects (physical appearance and flight characteristics) compared to perfection, with a 10 being perfect. (Not too many people here are qualified to properly evaluate an EA, but that's another story.) 10s should not be handed out as a way of saying "Thank you." That is one of the biggest problems of the ratings system. Too many users think giving 10s are a way to send a Thank You card.

[willsonman]

I think that this is a fantastic response from KE. Thank you Ryan. While we all want to just jump in a suggest solutions to the problem that we all have our own opinion on... lets mull it over a bit and revisit this in a few months in a more constructive way than in the past. Rather than "the rating system suxorz fix it or I'm gone" attitude I would suggest we start a suggestion box similar to the parts repository. Just dump our ideas without bickering. Maybe construct a poll to get some data points in a more concise manner to get the ball rolling a bit better. To me its a burned out topic right now to start this. Too many are riled up ready to shout "eureka" to this ongoing problem.

[abaser]

I couldn't agree more.

[Boof69]

Frankly I know the change is a simple one. Fixed in minutes. This is just a carefully worded political statement. What they really mean is they won't change a thing. There was talk about moving the swaps to be a share feature within the sim. Probably the best solution you could ask for.

[jeffpn]

I've said this before - the only ratings system that would be effective is not practical, and therefore can never be implented. The only way ratings can be accurate and reflective of the quality of the file is if a small number of qualified members were the only ones who could rate files. They would have to be unbiased towards any user, and they would have to rate every file. Obviously that can not happen, so to me, it doesn't really matter what gets done to the rating system, if anything. This is why I don't care what people say about our (Maj. and my and whoever does the physics) models, or what number they assign to them. The ratings is a meaningless system, and always will be. I model because I enjoy modeling. When we finish a model, it gets uploaded for others to use if they care to use them. Whether our models get all 10s, or all 1s, or 0 downloads, or 10,000 downloads will not affect my interest in the hobby of modeling. We can keep modeling and not share our models, but I prefer to allow anyone to use our models if they care to use them. This is why we upload. We do not upload for 10s or for download count. What else would we do with a finished model? Why not post them for the world to use? There's nothing else that can be done with them. If you like them, keep them. If you don't like them, delete them. No harm, no foul, don't care.

[Boof69]

How many times can you explain that you don't care about the ratings? :D

[Madratter]

I've said this before - the only ratings system that would be effective is not practical, and therefore can never be implented. The only way ratings can be accurate and reflective of the quality of the file is if a small number of qualified members were the only ones who could rate files. They would have to be unbiased towards any user, and they would have to rate every file. Obviously that can not happen, so to me, it doesn't really matter what gets done to the rating system, if anything. This is why I don't care what people say about our (Maj. and my and whoever does the physics) models, or what number they assign to them. The ratings is a meaningless system, and always will be. I model because I enjoy modeling. When we finish a model, it gets uploaded for others to use if they care to use them. Whether our models get all 10s, or all 1s, or 0 downloads, or 10,000 downloads will not affect my interest in the hobby of modeling. We can keep modeling and not share our models, but I prefer to allow anyone to use our models if they care to use them. This is why we upload. We do not upload for 10s or for download count. What else would we do with a finished model? Why not post them for the world to use? There's nothing else that can be done with them. If you like them, keep them. If you don't like them, delete them. No harm, no foul, don't care.

While I agree with much of what you just said, I think there is a bigger issue than just the ratings per se that has Boof riled up.

Now speaking just just for myself, the deal is this. Regardless of why we do this, the SWAP pages are a VERY large part of what makes RF better than the competition at this point. I believe that to be a fact, but maybe it is just a strong opinion held on my part.

Given that, there is a certain amount of neglect given to our (the modelers) community. Stuff that would be a huge help to us simply has not been made available. The gauges is one example of that. Another example is the documentation of things like nup values. You have to go hither and yon to find some of this stuff.

Now if we all went away, it wouldn't necessarily impact them much in the short run. But it sure would in the long run.

Throw us a few more bones.

When we ask for things like better nup documentation, give it to us. The time spent preparing it is nothing compared to all the free labor they are getting. Resources are probably very tight. I believe that. But come on, how much time can it take to prepare something like that?

[jeffpn]

Wow. You sound very much like someone else around here! I won't say who, but a day or two ago I was talking to someone on the phone who made the exact same points about throwing us some bones. :cool: ;) :DWhile I agree with much of what you just said, I think there is a bigger issue than just the ratings per se that has Boof riled up.

Now speaking just just for myself, the deal is this. Regardless of why we do this, the SWAP pages are a VERY large part of what makes RF better than the competition at this point. I believe that to be a fact, but maybe it is just a strong opinion held on my part.

Given that, there is a certain amount of neglect given to our (the modelers) community. Stuff that would be a huge help to us simply has not been made available. The gauges is one example of that. Another example is the documentation of things like nup values. You have to go hither and yon to find some of this stuff.

Now if we all went away, it wouldn't necessarily impact them much in the short run. But it sure would in the long run.

Throw us a few more bones.

When we ask for things like better nup documentation, give it to us. The time spent preparing it is nothing compared to all the free labor they are getting. Resources are probably very tight. I believe that. But come on, how much time can it take to prepare something like that?(And I agree with what you said, especially the NUP thingie.)

[Boof69]

Riling up....?

[Madratter]

Riling up....?

Maybe not. It is very tough to get that sort of thing right from reading peoples written words without the help of tone and body language. My apologies if I misspoke.

[willsonman]

I think KE has thrown us some pretty big bones over time. I had the biggest gripe over bugs and issues with the sim and every issue I have brought up to KE has been resolved. I consider that a big bone. Staying on top of their own software is a big deal. There are many software companies in the gaming industry who just flip you the bird and say deal with it. The game Rage is one of recent note. I think the kick-off for G6 where they literally handed out free copies of G6 to many top contributors here was a bone I never thought would be thrown. Staying on top of the latest max version to have a functioning plugin for us as well. Let us all also not forget that this forum is FREE to us. That means they fit the bill for us posting our gripes and keeping those gripes on file! Also, the search function is free. Honestly I have no complaints right now about KE and they way they treat me. I've not done an EA in some time but I continue to help others with physics questions. I would probably help more but Jeff seems to have the quick draw finger for a lot of things and is almost never wrong. No gripes. Lets have a little perspective here.

[Boof69]

Yes lets have some perspective.

[jeffpn]

Oops, I was set to orthogonal. :o

[Madratter]

I stand by what I wrote.

[Boof69]

What did you right?

[Madratter]

I'm not sure what's left.

[Boof69]

I have no idea what your talking about.

[Maj. Numbskully]

:confused: Can't you look?

[Boof69]

Well I'm sure he wrote a few things.

[Madratter]

I have no idea what your talking about.

I thought you were punning using right instead of write, to point out nothing I wrote was getting anything fixed (whether it needed to be or not).

Evidently, it was a simple misspelling. Anyway, I was punning back.

As for what I wrote, it was the stuff on valuing the modeling community a little more.

[Maj. Numbskully]

I feel all warm fuzzy and valued :)

[Madratter]

I feel all warm fuzzy and valued :)

You should Maj. You should. :)

[jeffpn]

You should Maj. You should. :)

Well duh! He gets to work with ME!!! :D

[Boof69]

Yup just noticed the typo. No hidden meaning or pun intended.

[jeffpn]

Our local newspaper had a groaner contest once. I sent in almost a dozen of my favorite bad one liner jokes. I really thought one of them would win, but no pun in ten did.

[Boof69]

WA WA WA waaaaaaa.:rolleyes::p

[td9cowboy]

Ryan these are just the EA's that are suspects for abuse since we went to RF6. This list does not include any files by Flex because they have been pulled. It represents possible abuse to over 10% of the really hard and freely given work here. This has been going on for years. How long would it take someone with the proper tools to look into who is downgrading these otherwise highly rated and praised files? If you need more examples to form a conclusive opinion about the person or persons involved, I'll volunteer the legwork to collect many more examples from G5, it's not really too hard to spot them. It shouldn't take a super detective or many long hours of investigation before you start to see a pattern form or someone really stands out.

We all appreciate what you did to clean up the mess that Flex was making but Flex wasn't really the problem. Flex going off was the result of the problem. Some times we are too quick to blame the system when bad stuff happens, it's far from perfect, but in this case, what has been causing all this turmoil and drama for all these years is a people or person problem. The only thing you guys did was to supply the tool. If you need some free help, I'm here every day.

http://www.knifeedge.com/forums/downloads.php?do=file&id=17088

http://www.knifeedge.com/forums/downloads.php?do=file&id=16239

http://www.knifeedge.com/forums/downloads.php?do=file&id=16226

http://www.knifeedge.com/forums/downloads.php?do=file&id=16149

http://www.knifeedge.com/forums/downloads.php?do=file&id=16120

http://www.knifeedge.com/forums/downloads.php?do=file&id=15940

[Junkboy999]

I stopped worrying about ratings when G 4 came out. People would rate a G4 EA with 1’s because they could not fly it in G3. Why would people that only have G3.5 Rate G4 files.

Rating systems suck. I lived through 20 year in the military where we tested, ( a score ) + physical fitness ( score ) and a Airman's performance RATING ( score ) . All three scores added up to compare you against all others fairly. RIGHT ( it mostly came down to who you know and who you ^%$#. )

Just stop worrying about them or turn them off.

Why do you go through this year after year?

G7 will be out soon... and guess what....Yatta Yatta Yatta Blah Blah Blah....


Sorry just had to edit this post to add Wow Really Jeff ? average? on all his file, then asking Ryan to remove your rating. They should be left just to show people what not to do. Childish

[12oclockhigh]

Just brain storming this...
1.) eliminate the cumulative average.
2.) create a system within the software whereby after having flown the particular plane for more than 1-5-10 hours, you are requested to rate the plane.
3.) have the software report back to the website the hours flown this month on contributor submitted content. Good planes will have a lot of hours flown off.

Yes I realize it is half baked... just some ideas thrown out.
,
,

[jeffpn]

It's not a bad idea, but I would bet it'd take alot more effort than KE is willing to budget.

[td9cowboy]

IMHO this particular problem has nothing to do with the quality of the uploaded file. It's a personal attack on the up loader. If the dirtbag gets lucky, sometimes he 5's it before it even shows a download.

[td9cowboy]

Let's assume for a moment that this is ONE very determined and sick individual that is using multiple ID's and IP's to cover his tracks. Through clever use of a loophole in the software and a good understanding of human nature he has been able to change this place forever. He has cost KE a lot of money in software sales and forum moderation costs. He's ruined peoples lives that loved to create content and share it here as a hobby. He's discouraged others from even posting their content on this site. Even though most users here are not personally affected by him, we are all suffering from the consequences as a community. I am deeply saddened that we appear to have lost yet another valued member and contributor to him. If anyone has any direct knowledge of such an individual please forward that information to KE to help them ferret him out. I'm willing to do all I can to help also.

[Rotorbroke]

Wow. I really hope you guys who work so hard to provide quality products to us takers, get taken care of fairly. I don't know if there is any really great system involving humans, input, personalities, hard workers marginal works, and thos who think everyone is equal and we should never hurt anyones feelings and lower standards to do that. I like to see ratings, but I assume most are honest. I think if you make an input, a name is not too much to ask, and no comment/no rating. Mature people can make this work. I think in mind, But I am a taker, not a giver, and have only provided marginal photos for people to volunarily try to please others and seem proud of a product. Just my 2 cents. But I hope you find a solution that works and is fair. Good luck....

[Maj. Numbskully]

Just brain storming this...

2.) create a system within the software whereby after having flown the particular plane for more than 1-5-10 hours, you are requested to rate the plane.
3.) have the software report back to the website the hours flown this month on contributor submitted content. Good planes will have a lot of hours flown off.

Yes I realize it is half baked... just some ideas thrown out.
,
,
I don't think its half baked at all !!
I had the same idea as #2 !
I even started to post it and then thought the same thing jeff does ...I think its just to much to implement .so I didn't
Idea # 3 is good to.

[csgill75]

Just brain storming this...
1.) eliminate the cumulative average.
2.) create a system within the software whereby after having flown the particular plane for more than 1-5-10 hours, you are requested to rate the plane.
3.) have the software report back to the website the hours flown this month on contributor submitted content. Good planes will have a lot of hours flown off.

Yes I realize it is half baked... just some ideas thrown out.
,
,

as far as #2 goes, 5-10 hours is a awful lot of time to spend flying ANY particular aircraft. I myself only use the sim when the weather is not good or its night time and i may spend 5 hrs a week in the sim flying 15-20 different models. If the model is not a good model I won't fly it period. I will also rate accordingly, sometimes with a comment depending on how bad it is and sometimes without if its not even worth my time since i know it won't be resolved.

Side note, Jeffpn, I remember you said once that the forums when loaded on a iPhone would not keep you logged in, well i have noticed the past couple days that I am always logged in on my iPhone and I wanted to know if you are experiencing this as well. Even when closing Safari completely.

[Maj. Numbskully]

I was thinking maybe after loading the aircraft the 3rd or 4th time

[jeffpn]

Csgill - I noticed a couple weeks ago I cannot log out on the iPhone either.

[abaser]

DROID RULES!!!! Logout when you want!:p

I'm up for an upgrade, so if there's any phone that can upload pics, let me know.:D

[Junkboy999]

To quote Td9cowboy

“ IMHO this particular problem has nothing to do with the quality of the uploaded file. It's a personal attack on the up loader. “

You hit the nail on the head. To keep on Quoting you

“ If anyone has any direct knowledge of such an individual please forward that information to KE to help them ferret him out. I'm willing to do all I can to help also. “

Can we hold you to that Td9cowboy? Pardon the Cowboy pun, but will you stick to your guns on that?

I my self have a life. I can not set at my computer or monitor a smart phone to see who is attacking who and for what reason or see if any one is attacking me.
I must of missed something. I know I will miss having Flex aground. I wish I knew what happen. I know it take two or more to have a fight. I can only question things.

Td9cowboy and other new here reading this thread, take 15 min and search the swaps for “Thegibson”. Look at the rating people gave his EA’s. Not just one, take the 15 min and
look at all his EA’s. Again all I can do is question why the forum moderator would remove some people comments and let other peoples comments stand. Draw your own conclusions.

I just ask, please Disable the rating system until a better system can be put into place, or human nature evolves to the point where we don’t feel the need to hide behind a computer
screen and hurt people because we're bored.

[12oclockhigh]

Junkboy999
I could not disagree more!

[Boof69]

Just so you know flex did what he did in the swaps and left all under his own power. All of it an overreaction. What got him completely worked up is an issue as old as the swaps. Am I the only one not thinking "Poor flex". If he was going to stay and cause more problems I say good riddance.

[jeffpn]

Sad to see him go (especially in a "I'm gonna take my ball and go home" sort of way - he deleted some important threads on his way out), but he did get bit by the same thing he was preaching against. After saying the forums don't need people who are here for an ego stroke, it seems he left for pretty much that reason. Ironic.

[Maj. Numbskully]

who said he went?

[Boof69]

I assume he did. He removed his threads and files and I haven't seen him signed in for two days.

[phrank]

Hmmm, Jimmie bans himself and Terry pops up out of nowhere.
Hey, whatcha trying to pull here? :p :D

[Madratter]

Hmmm, Jimmie bans himself and Terry pops up out of nowhere.
Hey, whatcha trying to pull here? :p :D

Well if he does pop up out of nowhere, it will be pretty obvious. He has a style that is hard to miss, both in posting and in terms of planes he models.

[jeffpn]

I know they used to work closely together. Trust me, they are not the same person. Right, Boof??

[phrank]

Well if he does pop up out of nowhere, it will be pretty obvious. He has a style that is hard to miss, both in posting and in terms of planes he models.

Heh! It was a lame attempt to ferret the old phart out of hiding. :D

I've worked with these two clowns before, they are definitely not the same guy. ;) ;)
(And this is the part where I blatantly violate forum rules by calling them names.)

[jeffpn]

Fun clowns, not scary clowns, right phrank? That's not name calling if you meant fun clowns!! :D

[Boof69]

I know they used to work closely together. Trust me, they are not the same person. Right, Boof??

I'm absolutely sure they are not.:cool:

[Maj. Numbskully]

Maybe he's me :eek:.....Dun Ta Taah
:p:p

[Ryan Douglas]

I only purged flexible's votes on the swap pages. Any removal of posts or swaps content is something he did himself. He is not banned and is welcome back, although it sounds like a little time away might be a positive thing for him.

td9cowboy, thanks for the links to suspected instances of ratings abuse. We aren't able to dive into it right away, but I am filing a link to that specific post (http://www.knifeedge.com/forums/showpost.php?p=232700&postcount=34), so if you do come across others, I suggest editing that post and keeping them in the same place. Thanks for the assistance. :)

[td9cowboy]

I only purged flexible's votes on the swap pages. Any removal of posts or swaps content is something he did himself. He is not banned and is welcome back, although it sounds like a little time away might be a positive thing for him.

td9cowboy, thanks for the links to suspected instances of ratings abuse. We aren't able to dive into it right away, but I am filing a link to that specific post (http://www.knifeedge.com/forums/showpost.php?p=232700&postcount=34), so if you do come across others, I suggest editing that post and keeping them in the same place. Thanks for the assistance. :)

Ryan, if you would give me a heads up through PM when someone gets freed up I'd be glad to give you lots to look at. Most of the active EA designers have migrated to RF6 but my bet is that the G5 5er/s will come back to the same IP/s.