As regular visitors have experienced firsthand, this forum is aging rapidly and will eventually reach a point where it's unfeasible to continue using it. The company that created the software, Discusware, has been out of business for 8 years now.
We're investigating the options for setting up a new forum and want to make sure we hear from you – the regular posters and lurkers who have sustained this welcoming community over the past 20 years. What features of this forum do you find most useful? What features of other forums do you wish we would adopt here? What features would help encourage growth without damaging the culture we already have (especially once the Wars of Light and Shadows series is DONE and ends up on new readers' To Be Read lists)?
Please share your thoughts in this thread. If you have trouble posting a reply, you can also email me at "brian AT urizone DOT net" and I will post on your behalf (with permission). There is no deadline on this survey – the new forum definitely won't appear overnight!
I'll start! I think…
A new forum should encourage new visitors to join. It should work on a variety of different browsers (including mobile phones). It should offer (but not require!) single sign-on using a visitor's alternate accounts (Google, Facebook, etc.). It should have a short, simple, understandable set of rules that reflect this community's culture.
A new forum should encourage visitors to stick around and post. It should be easy to format posts, add links or mentions (@s), or receive email notifications about topics of interest. It should offer some level of customization to visitors (themes, font sizes, avatars, profiles, etc.) without over-cluttering threads.
A new forum should be easy to administer. It should use modern programming languages and have regular feature / security upgrades available. We should be able to self-host it on paravia.com to avoid the content / data / privacy policies of third-party sites (Facebook, Google, Goodreads, Reddit, etc.). It should be easy to moderate users, posts, and spam.
Received by email from Roisin and posted with permission: Thanks for this post. Been lurking & posting on here since easily the Millenium-ish era, been reading Janny’s works since '97, and keen to follow this new development. The parts I’ve really enjoyed were:
The chat
The teasers
Getting to interact with Janny personally
In the earlier days I did enjoy the wide ranging discussions - but I also was burned up a bit by one post 9/11 - a lot of clap back from a perhaps unwise post about US servicemen & women that was then deleted and so have remained very much less engaged since then. Yet I never got into other forums like reddit or goodreads or amazon. I liked the uniqueness of paravia.com.
In that time, my username has changed a score of times as I moved to and fro and across the equator - as the accounts kept running out. I can’t even remember what I used to call myself…I’m nearing 50, and so things are becoming a little misty. I’d like to keep an original account and name - and just be known by that.
Also, second the issue with mobile phones. Right now I’m typing into a text box the size of a stamp and cannot make out the small print even when I use my shortsighted left eye and squint …
Security is key, I don’t want the data mining stuff - but will not be able to have that if all those tie-ins to known data mining platforms?
I think there should be a site specific login with alternate signin options for those who prefer them. Various folks have issues with different social companies, often with good reason.
Also does the current forum have an ongoing cost to use and maintain? What available options are no more expensive than the current system; or free? I don't have an answer for now, but it's something to consider.
Long time lurker, infrequent poster here coming out of the woodwork to add their Aussie 5c (since we don't have 2c coins anymore! )
I'd absolutely and heartily agree that having a forum-like service hosted and securely kept here at Paravia.com would be the most desirable option, as externally-hosted or controlled/owned options often subject the board owner and moderators to certain levels of data storage / saving permission requirements, and a particular degree of content ownership rights that really wouldn't suit an author's privacy & IP requirements (imo)
phpBB boards can be good, though the format can be clunky and unwieldy to set up and maintain (unless a basic installation is maintained without any unnecessary plugins that break with updates)
Having a mobile-compatible forum would be cool, but that kinda limits forum-specific softwares that one could look at as being responsive…
Another idea could be to have a private (member only / password protected?) WordPress subdomain rather than a "forum" as such, and then the separate parent posts could act like the posts here, and the comments section could function similarly, with the added benefit of threaded conversations and replies.
Then, you could have a singular Page as the "Index", and I believe there would be a Javascript or Ajax plugin that might be able to auto-update that page with new topics that are posted.
And with WordPress, the admin and mod teams can assign user rights and privileges, allowing a certain level of control over spam-posters etc. with a system like "must reply to 5 posts before you can post your own original post" - or the posting rights can be limited to different sub-sections of the WordPress blog/forum.
Actually, now I think about it, there's a community/forum addon to WordPress called BuddyPress that might be an option?
We could even design the WP layout to function/look similar to this existing forum. I'd be happy to help with that, though my skills have their limits
But I agree with Roisin in that having been here for the better part of 20 years and being able to chat to other fans of Janny's work, and speak with Janny directly as well, has been such a wonderful experience. Would love to keep that!
In thinking about how to answer the query and proffer my thoughts as a long time user of this forum, to me these are the important things for the forum going forward.
#1. Having Janny as a user, offering sneak peaks and responding to direct queries. So it's not just a fan forum but one where the creator joins the conversations.
#2. Be a safe place for all to post - which will means some level of access control. similar to the comment below, this can be specific to the forum or use a third party authentication system
#3. Needs to be able to be simply administered. Whilst I understand the desire to host the chatboard for good reasons, this then also requires the administrators to upkeep the site, upgrade it, resolve tech issues, etc and do migrations like this when it's so old it's unsupportable. I also wonder the fact the forum is hosted on the website makes it more exclusive or harder to find than say using a hosted forum with wider advertising and cross promotion with say fans of Ray Feist?
Another community I'm a part of is intfiction.org relating to Interactive Fiction/Storytelling. They switched a few years ago to Discourse. (https://www.discourse.org/)
Oh, I love Discourse - I am a user on a forum set up for the users of the main software I use at work, and it is just so very clean and easy to use / maintain from what I've seen.
Like a very sleek and modern phpBB without all the php/mySQL stuffaround and web2.0 grayphics / tables limitations
If you like, I can get in touch with the tech team that set this forum up that I'm a member of and ask them how one would go migrating data from whatever this platform is called, and they'll get back to me pretty quick smart!
@David: Discourse is actually on the short list of solutions I'm researching because of its active development and its "civilized discussion" mantra. Funnily enough, the IF forum is where I've gone to observe Discourse features in action (I wrote Augmented Fourth and lurk there when time permits).
I'm looking at the other PHP heavyweights as well, but they tend to have a lot of intrinsic cruft and "the way we've always done it" baked in that might interfere with the ability to stay modern, stay pretty, and stay easy to upgrade.
@Laneth: We probably won't be migrating the Discus contents to a new forum (it's all stored in Perl-edited flat files), but there will be a concerted effort to preserve it for historical curiousity, hopefully with a modern interface. No need to reach out to your tech team but thank you for the offer!
@Brian - No stress at all mate, and thanks for letting us know the tech-side of things too!
The historical value of this repository of discussion and knowledge would be invaluable to folks who are new to the series and become as obsessed as some of us have been - imagine coming here after reading the series and finding all the speculation & information and having your imagination fire up all over again!
Keen to see the evolution of Paravia within our reality!
I've updated the Chat Area to make it more artistically compatible with how Janny's new website will look.
This was a mostly automated process since the Chat Area currently encompasses over 16,600 posts spanning 1400 hard-coded HTML files full of charming code from the Netscape Navigator era. Please let me know if anything isn't quite right and I'll fix the issue.
Unfortunately, this is just a visual band-aid and does nothing to fix the ongoing issue with making new posts in old threads and topics. If you experience posting problems, please try to use the Arc 5 topic area (where this post is located) until we can move on to a new forum software solution.
Gosh! I haven’t been here for a long time. I’m excited about all the new stuff and follow posts via email notice while I patiently wait for Song! Meanwhile I shiver for the country and pray that no one gets coronavirus. I’ve been in hibernation since March but don’t mind so much. I have taken up an old love - quilting - again so keep happily occupied doing that while I watch and listen for the final demise of our country. I’m not reading as much as I used to as its hard to read and quilt. Also at the moment real life seems more vital than fiction. Janny, I shudder about your crazy governor. I’m in Connecticut and grateful to be here. I’m living at my daughters and do not have to go out as ahe provides me with sustenance while I fill all other needs via the internet. I’m raging pissed about what Trump and his little hairlipped friend are doing to the post office!
I currently have a private instance of Discourse up and running so I can carefully assess different facets of operation: security, data policy, administrator experience, user experience, customization, etc. In addition to its sound technical profile and forum features, I appreciate Discourse's emphasis on community and civilized communication. Unless there are critical gaps or complaints, this will likely be the baseline software we adopt.
In parallel to my assessment, I'll be writing code to convert the ancient Discus threads into a simple data model, and even more code to import that data model into Discourse's database format. Priority will go to posts that future fans will find interesting or enlightening (for example, any thread where Janny provided her authorial input to a direct "You Asked…" question about her books or the writing process). Lonely unanswered single post threads, esoteric news threads from 2004 with broken links, and threads sharing ancient TV show recommendations will not necessarily make the cut (though of course there will still be zany off-topic areas to post in on the new forum!)
With the exception of Janny's own account and posts, user accounts will NOT be migrated. Old posts will still list the responsible username for context, but they will not be linked to whatever new username you create in the new forum. This decision is part selfish, to reduce the number of moving parts in the migration process for 17,000 old posts, and part "fresh start", marking a firm "right to be forgotten" line as we start something new.
Sometime in the coming months, the forum will be ready for public experimentation. We'll have a soft launch period early on where you all can reserve your favourite usernames before the new forum is directly linked to the world. I'll put up a migration guide so people that have used Discus forever understand how Discourse does things (slightly) differently. There will also be a regularly-wiped sandbox topic area where you can feel free to test posts out without worrying about screwing up a real thread. We'll have clear, simple copyright, data, privacy, and conduct policies approved by Janny to make sure we stay a friendly, inclusive space for everyone. Once everyone is comfortable with the soft launch, we'll retire Discus permanently and swap in the new software on the top menu bar across the website.
This is just the PLAN and I am just the IMPLEMENTER. You all have a stake in this community. Feel free to raise your concerns early and often, and we can work through them together!
Thanks for the update, it truly is appreciated. You're doing an absolutely fantastic job - the new site looks amazing! Thanks to all who've been involved
With regards to Discourse, if you'd like an alpha tester whose had some exposure/experience with the communities, I'm sure David and I would be willing to give you a hand as needed prior to the soft launch.
It makes sense to not be migrating user accounts - unless you had the ability to see user activity in the back end of the current setup, it would be a nightmare to have to determine whom to move and whom to purge / archive.
I imagine that you'll be leaving a redirect / migration page here where the old forums would be in the event that an old user returning from some distant past via a bookmark can be informed of the forum move?
This is all so very exciting - the spring clean energy that has come as a consequence and unexpected side-effect of this global paradigm shift is helping tidy up the dust and cobwebs that have clung to our regular haunts and processes for too long!
An update on the forum is long overdue. The forum migration went on the back burner last fall in order to support Janny as she simultaneously readied 5 separate books for the March e-book release while continuing to write Song of the Mysteries.
Now that those priorities are settled and I'm no longer distracted by my annual winter diversions (Puzzle Boat 7 and Advent of Code), I'm back to it! I've completed research into the various features and extension points in the Discourse forum software and will begin customizing the software to our community's needs and page styles this weekend. I'm aiming for a pre-launch beta test for eager testers in the Feb/Mar timeframe (it will be announced here), with a full switchover by May. This is a firm infant-driven deadline!
Discourse is wonderfully modern (and mobile-friendly), so if you are used to any other forums or communities created in the past decade (Reddit, Goodreads, Facebook, etc), you shouldn't have a problem getting acclimated. There are almost TOO many features offered out-of-the-box, so we may start with a more spartan experience and then gradually open up new features as people get comfortable. I still plan to make sure that everyone here gets first dibs on their preferred usernames and has time to get comfortable with the new forum with plenty of hand-holding – I do not want to lose anyone just because things have become different!
Also, the new forum will have a separate area for website-related posts like this one. I'm very aware that people are here to discuss Janny's books first and foremost, not posts about software updates and 0s and 1s. I have these posts in the Song of the Mysteries topic area for now solely because of the instability of this ancient software, so thank you for bearing with me until we have a reliable "geek" channel going forward.
Discourse is good, as has been stated. Being able to use markdown in posts would be a wonderful thing for expression, and it doesn't have to be visible in the UI. I would definitely agree that less is more and am 100% in favour of at least temporarily hiding features. Most people here would seem to value intelligent conversation over fake-reward social media mechanisms (another point in favour of using Discourse). I would guess the driving points for visitors would be:
1. finding answers to questions 2. discussing the books 3. hearing from the author 4. upcoming books and events
None of which require fancy features and could be hampered by such.
Two Exceptions: Since there is a really lovely wiki (thanks for that!) it might be worth the effort to make it so users can use simple [[Wiki_Page]] links in posts. It would also increase visibility and encourage more collaboration on the work. Most quesions could be answered there rather than endlessly repeated in posts. I didn't immediately see a generic plugin for custom markdown tags in posts and I'm an old school LAMP dev so writing one in RoR is not on. I do see an official spoiler tag plugin. Nice and possibly a base for wiki links.
Regular forum features and organization should cover the rest, but that can be explored in the beta when there is something to point at.
Additional Content Requests: - Paravia map link - ways to show support for the author and/or her favoured causes (also, it'd be nice if you could buy all short stories with one click)
Migration is hard. Both the task and the fact that people generally do not appreciate change, however necessary. Not a new thing, but in stating it perhaps it will be a tiny bit easier moving forward. It is certainly worthwhile. Thanks to all for the thoughtful posts and work on the upgrades. It is understandable that these updates would be here and in itself underlines the need for change. Plus, some of us actually enjoy the geeky bits!
Welcome here, Maia, and thank you for your thoughts!
The spoiler tag plugin is already installed on the new forum. No one will ever have to type SPOILER SPOILER SPOILER SPOILER at the beginning of their post ever again.
As of last night, the migration of Chat Area data into the new forum is complete. We could use additional sets of eyes on the migrated topics for quality control, if anyone has too much free time.
How can I help?
Compare topics against the old copies on this Chat Area and report problems that might have been introduced in transit (consistently funky formatting, many topics in the wrong category, etc.)
You can also report minor problems (one reply with weird characters in it, a title that no longer makes sense in the new categorization scheme) but these are less important.
This is purely a data migration check! Don't worry about styles or user experience yet.
You can report problems directly to me by email at brian AT urizone DOT net, so we don't clutter up the Chat Area and people's email digests.
What should I know before starting?
As part of the data migration, all topics were recategorized to the most logical location. The new forum is arranged around categories (the overarching series) and tags (one or more book titles). Instead of navigating 11 separate sub-forums, one for each Volume, you simply create a topic in the Wars of Light and Shadows category and tag your topic with the book you're discussing. This helps reduce visitor indecision (where do I post?!?), opens up discussions to more passing eyes, and allows people to filter on just the tags they care about.
Several topics from the early 2000s are missing their first few replies (probably due to ancient bugs in the Discus archiving bot). I opted to retain these topics in case there's valuable knowledge in what remains. You do not need to report these topics if their beginning is ALSO missing in the original Chat Area topic.
All topics can be viewed without login, so account registration is disabled. Since the database might need to rollback, I don't want people to have to create their accounts more than once.
The new forum may experience unannounced downtimes of up to 15 minutes in the event of a database rollback.
I will correct big, sweeping problems by rolling back the database and remigrating everything.
I will correct minor problems by hand through the new forum interface.
Once the data is fairly settled, we will move on to the next phase (probably in the next 1-2 weeks).
What are the other phases?
A beta test (2 - 3 weeks), where people can claim their favorite usernames and try out the forum. Emphasis on: improving user experience, visual style tweaks, and identifying the confusing parts so I can write a How-To Guide that helps other people make the transition.
A soft launch (1 - 2 weeks), where we stop posting in the Chat Area. The new forum becomes the authoritative one but we don't announce it beyond the regular visitors here until we're sure that all of the kinks are worked out.
A hard launch, where we archive and delete the Chat Area, redirect all links and bookmarks, and live in the future!
What are the technical details of the data migration? 669 topics containing over 16,000 replies made the cut. Automation was key to making this possible.
The starting Chat Area format was a Discus flat file. Each time someone made a new topic in the Chat Area, Discus created a new HTML file containing the topic, its replies, and all of the HTML/JavaScript needed to render the UI. I started with a one-click task that downloads thousands of files from the webserver and makes copies in a staging area.
Next, I wrote Java code that digs into each file, extracts the important parts (title, poster, text, date) and discards the clutter. This cleansing process also eliminates ancient links that return a 404, updates Discus emojis to Markdown emojis, chastises that one reply that had BLINK tags in it, hides any email address posted in the past, and then assigns new categories and tags.
Next, I wrote Java code that uses the Discourse API (via a REST call) to pull the clean data in. I can run it all at once or by category. Much of this work was already done in open-source software, so it was a matter of adapting rather than starting from scratch.
I ran the import one category at a time at first, since some obvious problems required newly cleansed data. For a few days, it was a continuous cycle of "clean, import, find a problem, rollback, repeat". Most of this cycle takes seconds to run. The import step takes less than a second per topic/reply, but requires a 1-minute cooldown every 200 posts to avoid the API throttle limit – so a full import takes about 3 - 4 hours.
I scaled the forum server up to an AWS t3.small as a result of the import because t3.micro (the bare minimum) just wasn't performant. This increases operating costs by about $4.50 per month.
While people are vetting the data migration, I'll be enhancing the import step to support "just new replies". This will be used before switchover to quickly catch any new posts without having to reimport everything.
) 