Model Pages planning

So if I’m interpreting the old URL correctly, the old system identifier looks like the primary key or at least an index in a (very large) DB table?
I suppose if that’s the case, the other information, username, email address, album title etc are all stored alongside - interesting though that the username searches on the old site, e.g." Tom Carson" yield zero results, however clicking the username on the album page and subsequently copying the revealed email address into the old search does return the results correctly …

So “snaps” can only be searched by email address and classification - does that complicate matters or am I just over-thinking things?

Thinking about it a little further, it’s probably because the search field page was designed to only look in the email address field, so probably not an issue.

Andrew,

You’re right about making it a Web app. In fact, that’s what I did for the lajagclub for online mail and Tales archiving. Just reviewed my code. The page index.php validates user access, then launches menu which is HTML and uses CSS to display the user interface. Each command launches an HTML file with embedded JavaScript to handle the PC based actions and a PHP file that handles the server side. It’s complicated, but I can reuse a lot of the code.

I think you would like it better if I process an entire Section, so all you would have to do is type in the Section name and click Transfer. All the files and a database for access to them would appear on the new site. All names, key words, date’s or whatever is available for an album or photo would be a field in the database. To address your idea to distinguish old from new, I’d just add another field, e.g., SOURCE with possible values “Old Site” or “New Site.”

Ed Sowell

'76 XJ-S coupe, red

http://www.efsowell.us

Larry,

“These are the links I’m referring to, however checking over archived posts post-migration, looks like these links have been stripped out of the posts already”

I have to confess that, preferring to receive post via email, I’ve never used the old-site forums. But from what you say, it doesn’t sound like the archived posts were migrated correctly. It’s probably like the situation when someone sends an Word created email with just links to photos instead of embedded. Since the link is still on the sender’s PC, the recipient can’t possibly see them. At some point, if someone reminds me, I’ll take a look at the source code for the old-site pages and see if that’s the problem. If it is, it should be possible to drag the needed material over to the new site.

Ed Sowell

'76 XJ-S coupe, red

http://www.efsowell.us

Ed,

Trust me, the forum post content is all there, exactly as it was on the old site. Every URL which points to an album, every tinyurl… everything. This includes the very first digest from back when the list first started as a mailing list (some of which are over 1 million characters long, so whenever a background process touches one it spits the dummy because the system limit is 32000 characters per post).

Once the photos albums are moved over then I can create a job which will replace the old URL to the new one within the body of the posts (all 1.8 million of them), but first we need to have the same content present on the static site.

I’d like to clarify that what I’ve signed up for is migrating the photo albums. Haven’t had time to fully understand the links stuff so don’t want to commit. But since, if I understand correctly, it’s the messages that refer to the photos, not the reverse, once the photos are migrated MySQL quires should be able to patch the new location under the links in the messages.

Ed Sowell

'76 XJ-S coupe, red

http://www.efsowell.us

Ed,

If you can come up with a way to duplicate all the albums which are in jag-lovers.org/snaps to the wordpress site then I can handle the links here on the forum side.

I suspect that you may have trouble with the anti spam measures on the old site.

Ok. I’ve had a bit of time to think about the photo migration and I know we want the new container (apparently a wordpress plugin) built to receive the data etc., but to my mind, because so much modification/search & replace has to be done on the strings, e.g.

http://www.jag-lovers.org/snaps/ snap_view.php3?id=988389343” to “https://jag-lovers.com/albums/tom_carson/Parkinson XK 120 Hot Rod”

surely it would be easier to do all the string changes before the move? I’m talking about working on a duplicate database here, not on the original, BTW. (Fix engine before waxing bonnet)

If it were a DB that I’d created, (I can only visualise it that way, not knowing the actual structure) seems to me that the first order of biz would be to re-label the old fields …& if further filtering were deemed necessary, add further granularity by increasing the number of fields. Of course I have no idea how many tables or queries are involved, so that may all be out of the question.

Using an “if index <= 25 then” or equivalent statement, one could run a string replacement routine on the first 25 records and run a few tests to see if the results returned correctly.

It’s pretty hard to know how to proceed without knowing the structure of the stored data though …

“So if I’m interpreting the old URL correctly, the old system identifier looks like the primary key or at least an index in a (very large) DB table?

Almost certainly several tables with “relations” between them. E.g., a smal table with a row for each Section. A table with a row for each album, fairly big. It has a “foreign index” field that that identifies an album. If Section 1 has 100 albums, that will add 100 records in the albums, all with 1 in the foreign key field. Finally a huge table with a row for every single photo, each keyed to a record in the albums table.

This structure should be retained in the migration, making as few changes to field names as possible. Becomes very confusing if you don’t.

Ed Sowell

'76 XJ-S coupe, red

http://www.efsowell.us

“For that post I would need to replace: “http://www.jag-lovers.org/snaps/ http://www.jag-lovers.org/snaps/%20snap_view.php3?id=988389343
snap_view.php3?id=988389343” with something like "https://jag-lovers.com/albums/tom_carson/Parkinson XK 120 Hot Rod https://jag-lovers.com/albums/tom_carson/Parkinson%20XK%20120%20Hot%20Rod "

My guess is Snap_view.php builds the html string you want. Here’s an example I wrote for the lajagclub website:

http://www.efsowell.us/ed/share/editPhotoLinkTableDOTphp.txt

Ed Sowell

'76 XJ-S coupe, red

http://www.efsowell.us

Ed,

Yes, snap_view.php builds the page on the fly as you view it (i.e. the old site isn’t really static html in the strict sense).

The problem is, you won’t have direct access to the SQL, neither on the .org site, nor on the wordpress site.

What we need is a robot which steps through the old site and creates a mirror image of the same on the new site.

I need a log of the URL for each album, and individual photo on the old site, as well as where it ended up so that I can massage the historical posts.

Larry,

“surely it would be easier to do all the string changes before the move? I’m talking about working on a duplicate database here, not on the original,”

No. It should be done during the move, so to speak. Andrew (or someone… don’t know who’s in charge of what here) types a URL into his browser. It throws up a simple dialog with any parameters needed, such as the source data and target URL of where the converted stuff is to be written, and a go button called “Transfer” or something. After some period, the new files and database that contain everything needed for the J-L user interface to access it. Nothing is written to the old site.

It’s tricky programming, but I’m pretty sure it will work based on other stuff I’ve done. BTW, the “searching for strings” is probably best done with an HTML parser.

Ed Sowell

'76 XJ-S coupe, red

http://www.efsowell.us

1 Like

Agreed. I know how to do that. What has to be done is to mimic what’s done in the pages. I.e., we build our own SQL access in a .PHP file. The same PHP file will have code to write the stuff to the new site server.

I’m now thinking about who should do what an when. I think it would be better if you let me do the albums/photos first, since neither of us can work on the links in messages without the photos. If I’m successful with that, we will know it works and can apply the same techniques to the messages. Either I can do it, or provide you with what I’ve learned so you can do it. It wouldn’t turn out well working in parallel. Tracks might not meet in the middle :slight_smile:

Ok, the way I see it we have 3 tasks:

Landing_place_setup: Someone who knows wordpress well enough to make choices about plugins, page Organisation and such.

Moving_Company: this is the role I see Ed filling. Not necessarily the actual moving of the contents, but the creation of the code which would accomplish this task.

Mother: Making sure that the other two talk nice to each other, and when it is all done, changing the URLS on the new forum (This would be me).

So… who’s a wordpress propellorhead?

Just reread the bit about not having access to the SQL. One thing I’ve not done before is reads stuff from one server and write it to another. The code I will be developing needs permissions on both servers. I will won’t be able to test my code without at least read access to the relevant directories at the old site.

I’d like to peer at the code used to make the connection to the database. I expect it’s in source code for the Archives page, but I can’t get too it. If I go to either the old site or the new site and click on Archives I am taken to the new site pages.

Tried finding archived pages but the Wayback Machine doesn’t display the results because of ‘robot.txt’ header or something.

Any ideas?

Nick or Gunnar would be your contact people for that.

Just scanned through the discussion so far and wanted to add a couple of comments:

The Snaps album is based on PHP, MySQL and JPEG-files.

Please don’t assume this is built in a very rational manner, so thinking in terms of a logical database structure is probably not going to result in a successful path forward.

What I would be thinking in terms of is running a PHP-script to built up a database on the old server in a format that will be easily importable to whatever WP plugin is selected. So that needs to be decided first of all, then analysed. That may or may not be a stumbling block.

Why a stumbling block? The photo plugins I’ve looked at so far store their photos in the main WP media area. I’m not sure this will be too happy to have an extra 100K photos dumped in it (albeit it with small filesizes).

Nick

Thanks, Nick. I much appreciate you guidance. I’m no DB expert, but have done some work with MySQL and SQLlite. I’m pretty confident I’ll be able to figure out what’s going on when I see the tables. I’ll first take a look at it with a MySQL browser of some kind. My hosting service has a rather awkward one, but I’ve managed to work with it before.

I’m also familiar with using PHP to do MySQL queries, pull data out, put data in, etc.

I know relatively little about WP, but What I have done is write code that pulls JPGs off the server and make them available in a browser… very crude roll-your-own album if you will.

Anyway, I still haven’t tried gaining access to the database on the old server. I mentioned it to Andrew and he said he’d see that I got such permissions my I’ve not followed through.

BTW, I assume you are one of those who set up the old site and kept it running all these years. I am very appreciative of that. I would never had been able to keep my '76 XJ-S after retirement in the late '90s without Jag-Lovers. Thanks!

Ed Sowell

Progress report

I’ve made some progress with migrating the photo albums. At the moment it’s just a rough check of the basic functionality, and I’ve only imported 150 albums (of the 11K ones on the old system). The main effort to date has been to get the comments over from the old system, and keep them searchable.

The test site is:

http://ec2-34-208-71-12.us-west-2.compute.amazonaws.com/

I haven’t generated any documentation, customised the software much, or figured out how to handle new uploads, but I thought I’d show some proof of concept. The 150 albums I’ve put in are all very old, so the pictures aren’t large, or high resolution. I’ll try and add some more recent ones.

Please try out the “Search Photos” widget.

The link takes me to ADW-Wordpress and a “nothing found” message…

john North