SuperPermsTest
SuperPermsTest
SuperPermsTest allows to test permission plugins that claim to support "SuperPerms" for their correctness and performance.
Introduction
Bukkit created a standard API for setting and retrieving permissions of players called "SuperPerms" and there are currently a handful of permissions plugins that claim to support that API. This plugin runs testcases (see the Test Details page) against such a plugin on the server, trying to spot faulty implementations, misconfigurations or performance problems with "SuperPerms".
For Server Admins
If you are just interested in the results of the testcases, head over to the Test Results section and take a look at them. Those results are from my home PC (4 core AMD 2.8 GHz, 4 GB Ram) and while your server will likely produce different times, the relation between the times should stay roughly the same.
If you want to run these tests yourself against the permission plugin you currently use on your server, you should go to the Test Setup section and read about what you'll have to do to successfully run these testcases (you'll have to choose a player for testing and give him some permission nodes). You may spot performance or configuration problems on your server that way.
For Plugin Developers
If your plugin uses "SuperPerms", you may want to take a look at the Test Results section. It will show you some expected times for how long a call to "player.hasPermission(...)" may take under certain circumstances. You can also setup the test plugin on your test server by following instructions on the Test Setup page to make sure that the permissions plugin you use for testing your plugin works correctly and therefore can be sure it is (not) your fault if something doesn't work.
For Permissions Plugin Developers
If you plan on supporting "SuperPerms" or to make drastic changes to your implementation, running the testcases provided by "SuperPermsTest" can help you to spot errors early, instead of waiting for people to give you obscure error reports like "it doesn't work. Fix it!". Also, you may use the test results on the Test Results page to convince people that your plugin works correctly, is fast enough etc. and that it has to be an user error that's causing the problems (e.g. misconfiguration).
Some other info
I don't claim that these testcases are complete, meaning that a plugin passing all testcases is not equal to it implementing SuperPerms 100%. But it means that most of the practical usecases of "SuperPerms" are covered.
The project is released under MIT license. It is open source and you can find it on github here: SuperPermsTest.
I have a twitter account, which I use to announce new versions of my plugins (mostly): 
Comments
-
_ForgeUser7459264 Owner
-
View User Profile
-
Send Message
Posted Nov 20, 2011@Sleaker
Feel free to. I won't be able to work on the plugin until Tuesday anyway, so there'll definitely be no merge conflicts. I've pulled the pull request you setup (thought I had done it already).
-
View User Profile
-
Send Message
Posted Nov 20, 2011@Evenprime85
want me to add the isPermissionSet() test into my checks?
Also I'll note that the reason why SuperPerms doesn't suffer from recursion depth issues is because it flattens the permission structure when dumping permissions onto another permission, so all permissions only end up being a maximum of 2 steps away. This is why you see 2 different values 120ish nanoseconds and 350ish nanoseconds for most recursive permission tests.
Access time on PEX is pretty linear based on recursion depth if you notice the graph correlation. Which is expected for how it handles the permissions. (And from experience having a depth of 10 isn't uncommon - all it takes it creating a 10 group track and having each successive group inherit the last)
It is also by far the most confusing to try and get node negation working on. as Order very much matters.
-
View User Profile
-
Send Message
Posted Nov 19, 2011I've got some ideas and code from Sleaker and found myself some other things that are testworthy too, but currently have no time to actually write those things and run the test(s). I'll do it next week, probably tuesday. Promised.
Upcoming will be:
- case (in)sensitivity tests
- testing of the "isPermissionSet(...)" method, which is part of the SuperPerms API
- optional Permissions 2/3 tests
-
View User Profile
-
Send Message
Posted Nov 18, 2011@t3hk0d3
Not in the superperms spec - node.* is designed to give all desirable nodes (but some nodes might be meant to be false)
NoCheat has things like this.
-
View User Profile
-
Send Message
Posted Nov 14, 2011@t3hk0d3
Technically "SuperPerms" doesn't support wildcards, so the default behaviour is treating anything ending with ".*" as a normal string just like all others - meaning if children are defined, they get assigned, if not then not. That is in the cases where e.g. "plugin.admin.*" was explicitely defined by the plugin author in plugin.yml - if it wasn't defined there, then it is no SuperPerms permission and behaviour is undefined (and not tested). That's also where the difference between the plugins come from. For the other plugins the .* is considered nothing special if it is defined as a SuperPerm (despite them having a ".*" wildcard mechanism in their plugins).
But I see what you mean and will in the next run rephrase that part, label it not as a failure but as a deviation from what pure SuperPerms would produce in those cases, so people know to watch out for that when switching from or to PermissionsEx.
@jarrgoon
I'll do that, but I don't think there will be any differences, because (afaik) all plugins with database backend keep all permissions in memory or use the builtin storage for player permissions. Anything else would be likely very bad for performance (a database query will usually be much slower than e.g. a hashtable lookup).
-
View User Profile
-
Send Message
Posted Nov 14, 2011Thank you for good measuring tool :)
I have some comments on Test 11 and Test 12. They are seems to be invalid.
This is specially coded behavior, made by user request and discussed on IRC. I don't see anything wrong in this behavior.
Actually this is test for NOT supporting wildcards. Just think - if you have wildcard (node.*), isn't that implies what you want all subnodes (node.abc, node.abc.def, etc)
-
View User Profile
-
Send Message
Posted Nov 12, 2011I love what you're doing here. I'd like to see a comparison using a MySQL backend, especially to see if it improves the PEX results at all.
-
View User Profile
-
Send Message
Posted Nov 11, 2011New version v0.2 is now uploaded, waiting for approval. It has three new checks (and I removed one existing in return), and finally differences between the plugins start to show.
PermissionsEx 1.16 fails two of the three new checks.
-
View User Profile
-
Send Message
Posted Nov 11, 2011@Sayshal
I don't think I'll add that after all. Just makes things more complicated for me than I expected. Currently waiting for MC 1.0 to see what else breaks and then we'll see.
-
View User Profile
-
Send Message
Posted Nov 11, 2011@Evenprime Great to see you expanding from just NoCheat! Any updates on NoCheat Regions? :D